Jul 25 00:35:45.160823 ip-10-0-141-233 systemd[1]: Starting Kubernetes Kubelet...
Jul 25 00:35:45.598700 ip-10-0-141-233 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:35:45.598700 ip-10-0-141-233 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 25 00:35:45.598700 ip-10-0-141-233 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:35:45.598700 ip-10-0-141-233 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 25 00:35:45.598700 ip-10-0-141-233 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 00:35:45.601607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.601526    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 25 00:35:45.605431 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605417    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.605431 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605431    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605435    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605438    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605441    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605443    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605446    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605448    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605451    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605453    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605457    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605460    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605462    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605465    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605467    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605470    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605473    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605475    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605478    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605481    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605483    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.605492 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605486    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605488    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605492    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605497    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605500    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605505    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605519    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605522    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605525    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605528    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605531    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605533    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605536    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605538    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605541    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605543    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605546    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605548    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605551    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.605969 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605553    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605555    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605558    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605561    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605563    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605566    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605568    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605571    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605573    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605575    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605578    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605580    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605583    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605586    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605588    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605591    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605594    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605596    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605598    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605601    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.606406 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605603    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605607    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605610    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605613    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605616    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605618    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605621    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605623    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605626    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605628    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605631    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605637    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605639    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605642    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605644    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605647    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605649    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605652    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605654    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.606953 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605657    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605659    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605661    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605664    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605667    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605670    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605674    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605676    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605679    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605681    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605684    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605686    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605690    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605692    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605695    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605697    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605700    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605702    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605704    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.605707    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.607403 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606367    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606373    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606376    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606378    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606381    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606384    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606387    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606389    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606393    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606395    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606398    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606415    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606420    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606424    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606429    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606433    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606435    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606438    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606441    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606444    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.607911 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606446    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606449    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606451    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606454    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606456    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606459    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606461    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606464    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606466    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606476    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606479    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606482    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606484    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606486    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606489    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606492    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606494    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606497    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606501    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.608387 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606503    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606536    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606539    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606542    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606544    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606548    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606552    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606555    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606557    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606560    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606562    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606566    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606569    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606571    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606574    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606577    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606579    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606583    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606586    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.608853 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606588    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606591    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606593    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606596    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606600    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606604    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606607    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606610    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606613    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606616    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606619    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606622    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606624    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606628    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606630    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606633    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606636    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606639    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606642    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.609291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606645    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606648    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606650    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606653    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606656    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606659    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606661    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606664    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606666    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606669    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606671    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606674    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606676    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606680    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606683    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606686    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606690    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606693    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606696    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606698    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.609845 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606701    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.606703    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607140    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607146    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607149    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607152    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607155    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607158    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607163    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607166    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607169    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607171    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607174    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607177    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607180    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607182    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607185    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607187    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607190    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607193    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.610320 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607195    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607198    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607201    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607203    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607206    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607208    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607211    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607213    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607215    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607218    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607222    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607226    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607229    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607232    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607235    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607238    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607241    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607243    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607246    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607250    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.610843 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607254    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607256    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607259    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607262    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607264    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607267    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607269    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607272    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607274    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607277    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607280    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607282    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607284    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607287    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607289    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607292    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607294    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607297    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607299    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607302    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.611323 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607304    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607307    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607309    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607312    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607314    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607317    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607319    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607321    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607324    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607327    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607329    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607331    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607337    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607341    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607343    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607346    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607349    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607352    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607354    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.611902 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607357    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607360    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607362    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607365    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607369    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607372    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607375    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607378    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607381    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607384    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607387    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607390    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607392    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607395    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607398    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607400    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607403    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607406    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607408    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.612349 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607411    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607414    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.607416    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607483    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607491    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607499    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607520    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607528    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607534    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607539    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607543    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607546    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607549    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607552    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607555    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607558    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607561    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607564    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607567    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607570    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607572    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607577    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607580    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 25 00:35:45.612809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607583    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607586    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607589    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607593    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607596    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607599    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607602    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607605    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607607    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607611    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607614    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607617    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607621    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607624    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607627    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607632    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607635    2574 flags.go:64] FLAG: --enable-server="true"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607638    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607643    2574 flags.go:64] FLAG: --event-burst="100"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607646    2574 flags.go:64] FLAG: --event-qps="50"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607649    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607652    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607655    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607659    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 25 00:35:45.613323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607662    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607664    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607668    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607671    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607673    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607676    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607679    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607682    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607685    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607688    2574 flags.go:64] FLAG: --feature-gates=""
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607692    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607695    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607699    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607702    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607705    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607709    2574 flags.go:64] FLAG: --help="false"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607711    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607728    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607730    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607733    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607737    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607740    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607743    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 25 00:35:45.613906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607747    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607750    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607753    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607756    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607759    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607762    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607765    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607768    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607771    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607773    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607776    2574 flags.go:64] FLAG: --lock-file=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607779    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607782    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607784    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607789    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607792    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607795    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607798    2574 flags.go:64] FLAG: --logging-format="text"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607802    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607805    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607808    2574 flags.go:64] FLAG: --manifest-url=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607811    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607815    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607817    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607821    2574 flags.go:64] FLAG: --max-pods="110"
Jul 25 00:35:45.614543 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607824    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607827    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607830    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607833    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607836    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607839    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607841    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607849    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607852    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607855    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607858    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607861    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607867    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607870    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607873    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607876    2574 flags.go:64] FLAG: --port="10250"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607879    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607881    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06fae6e0fa432b0b1"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607884    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607887    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607890    2574 flags.go:64] FLAG: --register-node="true"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607893    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607902    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607904    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 25 00:35:45.615196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607907    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607910    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607917    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607921    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607924    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607926    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607929    2574 flags.go:64] FLAG: --runonce="false"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607932    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607935    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607938    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607941    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607944    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607946    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607949    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607952    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607956    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607959    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607962    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607964    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607967    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607971    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607973    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607978    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607981    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607984    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 25 00:35:45.615838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607988    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607991    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607993    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607996    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.607999    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.608002    2574 flags.go:64] FLAG: --v="2"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.608005    2574 flags.go:64] FLAG: --version="false"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.608009    2574 flags.go:64] FLAG: --vmodule=""
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.608012    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.608017    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608124    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608127    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608130    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608133    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608136    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608139    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608141    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608144    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608146    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608149    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608151    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608154    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.616424 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608157    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608159    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608162    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608164    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608167    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608169    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608173    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608175    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608178    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608180    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608183    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608185    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608188    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608190    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608193    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608195    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608198    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608200    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608203    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608207    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.616950 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608209    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608212    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608214    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608217    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608219    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608222    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608226    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608229    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608232    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608234    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608237    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608240    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608244    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608247    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608250    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608253    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608256    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608258    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608261    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.617415 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608265    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608268    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608271    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608274    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608277    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608280    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608283    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608286    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608289    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608291    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608294    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608296    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608300    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608303    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608305    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608308    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608310    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608313    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608315    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608318    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.617897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608320    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608322    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608325    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608328    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608330    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608333    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608336    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608338    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608340    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608343    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608345    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608348    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608350    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608353    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608356    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608358    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608360    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608363    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608365    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608367    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.618369 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608370    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608372    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608374    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608377    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608380    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608383    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608385    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.608388    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.609114    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616128    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616142    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616161    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616167    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616220    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616225    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.618897 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616228    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616231    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616234    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616237    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616240    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616242    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616245    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616248    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616250    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616253    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616255    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616258    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616260    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616263    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616265    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616268    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616270    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616273    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616275    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616277    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.619291 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616280    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616283    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616285    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616288    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616290    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616292    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616295    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616299    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616302    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616305    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616307    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616310    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616312    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616315    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616318    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616322    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616325    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616328    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616332    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.619802 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616336    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616338    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616341    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616345    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616348    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616351    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616354    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616356    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616360    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616362    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616365    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616368    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616370    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616373    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616375    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616378    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616380    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616383    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616385    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.620264 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616388    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616391    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616394    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616396    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616398    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616401    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616404    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616406    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616409    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616411    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616414    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616416    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616419    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616421    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616424    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616426    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616429    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616431    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616434    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616437    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.620714 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616439    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616442    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616445    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616447    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616450    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616463    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616466    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616469    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616472    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616474    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616477    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616479    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616482    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616485    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616488    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616491    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616493    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616496    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.621234 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616498    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616503    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616623    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616627    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616630    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616633    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616636    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616638    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616641    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616643    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616646    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616648    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616651    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616653    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 00:35:45.621746 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616656    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616659    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616661    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616664    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616666    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616669    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616672    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616674    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616676    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616679    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616682    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616684    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616686    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616691    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616694    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616696    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616699    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616702    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616704    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 00:35:45.622075 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616707    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616709    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616712    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616714    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616717    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616719    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616722    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616724    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616727    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616729    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616732    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616734    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616736    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616739    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616742    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616744    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616747    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616749    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616752    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 00:35:45.622560 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616755    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616759    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616762    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616764    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616767    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616770    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616772    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616776    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616779    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616782    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616785    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616789    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616791    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616794    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616797    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616800    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616802    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616805    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616807    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616810    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 00:35:45.623015 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616812    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616814    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616817    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616819    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616822    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616824    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616827    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616829    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616831    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616834    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616836    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616838    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616841    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616844    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616846    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616849    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616851    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616854    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616856    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616858    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 00:35:45.623519 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616861    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616864    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616866    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616869    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616872    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616874    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616877    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616879    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:45.616881    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616886    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.616976    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.619138    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.620823    2574 server.go:1019] "Starting client certificate rotation"
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.620921    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 00:35:45.624009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.620952    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 00:35:45.624432 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.623594    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 00:35:45.629123 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.629102    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 00:35:45.644370 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.644354    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 25 00:35:45.646824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.646809    2574 log.go:25] "Validated CRI v1 image API"
Jul 25 00:35:45.647636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.647617    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 25 00:35:45.651065 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.651040    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 80e591a5-e23e-425f-b116-636377fb9a71:/dev/nvme0n1p3 f7b06af3-a326-40cd-bf9e-119109e154a6:/dev/nvme0n1p4]
Jul 25 00:35:45.651133 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.651065    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 25 00:35:45.652771 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.652753    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 00:35:45.656111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.656007    2574 manager.go:216] Machine: {Timestamp:2026-07-25 00:35:45.654632545 +0000 UTC m=+0.366050936 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3162811 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22dc71e3753c4c277d4b3ee7bc9748 SystemUUID:ec22dc71-e375-3c4c-277d-4b3ee7bc9748 BootID:6e5adbda-b96c-4bbf-9bb5-fc527ce537a7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:a3:9e:c3:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:a3:9e:c3:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:4c:b0:40:26:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 25 00:35:45.656111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.656107    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 25 00:35:45.656241 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.656175    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 25 00:35:45.659663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.659640    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 25 00:35:45.659796 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.659666    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 25 00:35:45.660478 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.660469    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 25 00:35:45.660527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.660481    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 25 00:35:45.660527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.660488    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 00:35:45.660527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.660501    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 00:35:45.661813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.661804    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 25 00:35:45.664641 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664630    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 00:35:45.664752 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664742    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 25 00:35:45.664794 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664771    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 25 00:35:45.664794 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664779    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 25 00:35:45.664794 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664793    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 25 00:35:45.664902 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664803    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 25 00:35:45.664902 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.664812    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 25 00:35:45.665110 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.665099    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 00:35:45.665145 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.665117    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 00:35:45.667423 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.667405    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 25 00:35:45.670400 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.670383    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 25 00:35:45.670465 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.670430    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 25 00:35:45.674123 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674109    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674127    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674133    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674138    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674148    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674156    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674164    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674169    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674175    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 25 00:35:45.674179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674180    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 25 00:35:45.674434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674189    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 25 00:35:45.674434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674194    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 25 00:35:45.674434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674218    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 25 00:35:45.674434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.674226    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 25 00:35:45.677127 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.677114    2574 server.go:1268] "Started kubelet"
Jul 25 00:35:45.677205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.677171    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 25 00:35:45.677259 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.677192    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 25 00:35:45.677259 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.677254    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 25 00:35:45.677937 ip-10-0-141-233 systemd[1]: Started Kubernetes Kubelet.
Jul 25 00:35:45.679599 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.679578    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 25 00:35:45.679753 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.679711    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 25 00:35:45.680435 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.680409    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 25 00:35:45.680532 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.680415    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 25 00:35:45.680532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.680466    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 25 00:35:45.684784 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.684762    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 25 00:35:45.688038 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.686943    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-233.ec2.internal.18c5613dde8f0e09  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-233.ec2.internal,UID:ip-10-0-141-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-233.ec2.internal,},FirstTimestamp:2026-07-25 00:35:45.677086217 +0000 UTC m=+0.388504606,LastTimestamp:2026-07-25 00:35:45.677086217 +0000 UTC m=+0.388504606,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-233.ec2.internal,}"
Jul 25 00:35:45.689797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.689778    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 25 00:35:45.689797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.689789    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 25 00:35:45.689980 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.689863    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:45.690540 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690488    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 25 00:35:45.690652 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690488    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 25 00:35:45.690736 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690723    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 25 00:35:45.690736 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690662    2574 factory.go:55] Registering systemd factory
Jul 25 00:35:45.690839 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690748    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 25 00:35:45.690892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690863    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 25 00:35:45.690892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690871    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 25 00:35:45.691004 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.690991    2574 factory.go:153] Registering CRI-O factory
Jul 25 00:35:45.691061 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.691008    2574 factory.go:223] Registration of the crio container factory successfully
Jul 25 00:35:45.691061 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.691054    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 25 00:35:45.691129 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.691080    2574 factory.go:103] Registering Raw factory
Jul 25 00:35:45.691129 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.691096    2574 manager.go:1196] Started watching for new ooms in manager
Jul 25 00:35:45.691585 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.691571    2574 manager.go:349] Starting recovery of all containers
Jul 25 00:35:45.691753 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.691718    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 25 00:35:45.691849 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.691826    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 25 00:35:45.701813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.701794    2574 manager.go:354] Recovery completed
Jul 25 00:35:45.703126 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.703102    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hx7nm"
Jul 25 00:35:45.707047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.707035    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:45.708894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.708875    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:45.708955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.708900    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:45.708955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.708912    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:45.709371 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.709355    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 25 00:35:45.709371 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.709369    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 25 00:35:45.709455 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.709383    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 00:35:45.709455 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.709404    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:45.710150 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.710137    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:45.710195 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.710162    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:45.710195 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.710174    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:45.710335 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.710321    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hx7nm"
Jul 25 00:35:45.712426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.712415    2574 policy_none.go:49] "None policy: Start"
Jul 25 00:35:45.712478 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.712430    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 25 00:35:45.712478 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.712441    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 25 00:35:45.790149 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.790130    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:45.865489 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.865440    2574 policy_none.go:47] "Start"
Jul 25 00:35:45.900827 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.890573    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:45.900827 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.897265    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-233.ec2.internal\" not found" node="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:45.902033 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902022    2574 manager.go:337] "Starting Device Plugin manager"
Jul 25 00:35:45.902094 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.902051    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 25 00:35:45.902094 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902060    2574 server.go:87] "Starting device plugin registration server"
Jul 25 00:35:45.902281 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902271    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 25 00:35:45.902318 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902284    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 25 00:35:45.902389 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902375    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 25 00:35:45.902463 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902452    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 25 00:35:45.902557 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.902463    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 25 00:35:45.903045 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.903027    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 25 00:35:45.903107 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.903081    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:45.924835 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.924810    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 25 00:35:45.925911 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.925893    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 25 00:35:45.926000 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.925937    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 25 00:35:45.926000 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.925959    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 25 00:35:45.926107 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:45.925996    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 25 00:35:45.928472 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:45.928452    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:35:46.002891 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.002863    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:46.003891 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.003876    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:46.003989 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.003909    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:46.003989 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.003922    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:46.003989 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.003954    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.016281 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.016261    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.016343 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.016284    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-233.ec2.internal\": node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.027093 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.027069    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"]
Jul 25 00:35:46.027198 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.027139    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:46.027961 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.027946    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:46.028018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.027976    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:46.028018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.027990    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:46.030296 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.030285    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:46.030405 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.030392    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.030444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.030424    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:46.030992 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.030972    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:46.031069 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.031007    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:46.031069 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.031017    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:46.031069 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.030977    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:46.031162 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.031073    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:46.031162 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.031084    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:46.033177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.033160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.033269 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.033187    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 00:35:46.033850 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.033834    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 00:35:46.033934 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.033864    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 00:35:46.033934 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.033877    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeHasSufficientPID"
Jul 25 00:35:46.044698 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.044684    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.060153 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.060131    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-233.ec2.internal\" not found" node="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.064367 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.064353    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-233.ec2.internal\" not found" node="ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.092609 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.092589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.092660 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.092612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.092660 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.092633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b454936a2c3aaec87ddb9b7f44eca43-config\") pod \"kube-apiserver-proxy-ip-10-0-141-233.ec2.internal\" (UID: \"9b454936a2c3aaec87ddb9b7f44eca43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.144783 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.144735    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.193312 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.193388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.193388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.193450 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b454936a2c3aaec87ddb9b7f44eca43-config\") pod \"kube-apiserver-proxy-ip-10-0-141-233.ec2.internal\" (UID: \"9b454936a2c3aaec87ddb9b7f44eca43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.193450 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193414    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b454936a2c3aaec87ddb9b7f44eca43-config\") pod \"kube-apiserver-proxy-ip-10-0-141-233.ec2.internal\" (UID: \"9b454936a2c3aaec87ddb9b7f44eca43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.193450 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.193444    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6a7dffde4427b22d8c34d040f4ff0c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal\" (UID: \"b6a7dffde4427b22d8c34d040f4ff0c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.245406 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.245388    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.345990 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.345952    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.363154 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.363124    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.368527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.368205    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:46.446759 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.446677    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.550415 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.550386    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.619779 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.619763    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 25 00:35:46.620131 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.619908    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 00:35:46.652078 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.652049    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.697552 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.697484    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:35:46.717478 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.717445    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-24 00:30:46 +0000 UTC" deadline="2028-05-01 21:15:52.318840092 +0000 UTC"
Jul 25 00:35:46.717568 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:46.717477    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15524h40m5.601366834s"
Jul 25 00:35:46.752807 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.752788    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.854315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.854279    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:46.955206 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:46.955143    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:47.055656 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.055630    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:47.057776 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:47.057749    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b454936a2c3aaec87ddb9b7f44eca43.slice/crio-c97ebeb5b77fdfe7ad54203ca4cdb2946405de405647d436d1fd5732232d27b5 WatchSource:0}: Error finding container c97ebeb5b77fdfe7ad54203ca4cdb2946405de405647d436d1fd5732232d27b5: Status 404 returned error can't find the container with id c97ebeb5b77fdfe7ad54203ca4cdb2946405de405647d436d1fd5732232d27b5
Jul 25 00:35:47.058213 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:47.058194    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a7dffde4427b22d8c34d040f4ff0c1.slice/crio-74eeaca499dd60f89c1e17e23ac2e514b88b417193cd01bd28fae5fc0af7944b WatchSource:0}: Error finding container 74eeaca499dd60f89c1e17e23ac2e514b88b417193cd01bd28fae5fc0af7944b: Status 404 returned error can't find the container with id 74eeaca499dd60f89c1e17e23ac2e514b88b417193cd01bd28fae5fc0af7944b
Jul 25 00:35:47.064042 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.064029    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:35:47.124682 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.124665    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:35:47.156282 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.156256    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-233.ec2.internal\" not found"
Jul 25 00:35:47.222593 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.222490    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:35:47.291586 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.291554    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:47.305887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.305863    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 00:35:47.307040 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.307025    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal"
Jul 25 00:35:47.319031 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.319016    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 00:35:47.666188 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.666123    2574 apiserver.go:52] "Watching apiserver"
Jul 25 00:35:47.674035 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.674013    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 25 00:35:47.675515 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.675492    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-69jv9","kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf","openshift-dns/node-resolver-k5qjn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal","openshift-multus/multus-additional-cni-plugins-j5kdf","openshift-multus/multus-g9xqk","openshift-network-operator/iptables-alerter-jgzpl","kube-system/global-pull-secret-syncer-wf7xm","openshift-cluster-node-tuning-operator/tuned-8hswt","openshift-image-registry/node-ca-w5qb9","openshift-multus/network-metrics-daemon-wdr58","openshift-network-diagnostics/network-check-target-t8lqb","openshift-ovn-kubernetes/ovnkube-node-84xcm"]
Jul 25 00:35:47.678167 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.678143    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.680561 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.680511    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.680679 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.680656    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 25 00:35:47.680978 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.680942    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 25 00:35:47.681085 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.681039    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cn99w\""
Jul 25 00:35:47.681160 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.681109    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.681257 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.681234    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.683102 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.683082    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.683687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.683472    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.683687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.683490    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sv9bh\""
Jul 25 00:35:47.683687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.683510    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.683687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.683551    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 25 00:35:47.685272 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.685251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.685958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.685930    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.686246 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.686197    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 25 00:35:47.686246 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.686228    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vq97v\""
Jul 25 00:35:47.686775 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.686757    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.687614 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.687593    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.687704 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.687628    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.687704 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.687677    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vsx9v\""
Jul 25 00:35:47.688141 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.688051    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.690283 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.690264    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 25 00:35:47.690677 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.690654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.690995 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.690976    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.690995 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.690994    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9lhzw\""
Jul 25 00:35:47.691176 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.691159    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.691245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.691187    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 25 00:35:47.691245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.691236    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 25 00:35:47.691351 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.691264    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 25 00:35:47.692316 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.692030    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 25 00:35:47.693465 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.693443    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 25 00:35:47.693465 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.693463    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sp8wb\""
Jul 25 00:35:47.695393 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.694084    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 25 00:35:47.695393 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.695230    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.696096 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.695758    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:47.698313 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.698286    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:47.698419 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.698364    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:47.700853 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.700834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.701102 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-ovn\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.701200 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56kgv\" (UniqueName: \"kubernetes.io/projected/0c0b6892-c857-4590-8b2d-ca8b10d81751-kube-api-access-56kgv\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.701200 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-os-release\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.701200 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb4vr\" (UniqueName: \"kubernetes.io/projected/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-kube-api-access-nb4vr\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.701200 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-cnibin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701214    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-k8s-cni-cncf-io\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-bin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-etc-kubernetes\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59310f30-69f1-4d8b-8939-2320ae3ca7b4-host\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-socket-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-multus\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-system-cni-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-dbus\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.701496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-os-release\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-device-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-sys-fs\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-kubelet-config\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701593    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-kubelet\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-log-socket\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-env-overrides\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-socket-dir-parent\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-node-log\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-config\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-netns\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-tmp-dir\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-netns\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701811    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovn-node-metrics-cert\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.702047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-conf-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-multus-certs\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zbcw\" (UniqueName: \"kubernetes.io/projected/59310f30-69f1-4d8b-8939-2320ae3ca7b4-kube-api-access-9zbcw\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-bin\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.701996    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-cni-binary-copy\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702048    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-slash\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702141    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-registration-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-var-lib-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-hosts-file\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-systemd-units\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-script-lib\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8ppf\" (UniqueName: \"kubernetes.io/projected/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-kube-api-access-r8ppf\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702400    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.702813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-system-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702449    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpsp5\" (UniqueName: \"kubernetes.io/projected/90764f53-efa3-453c-9e7d-0a38b182318b-kube-api-access-jpsp5\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/59310f30-69f1-4d8b-8939-2320ae3ca7b4-serviceca\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-etc-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-hostroot\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-netd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cnibin\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-multus-daemon-config\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702813    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-659jg\" (UniqueName: \"kubernetes.io/projected/02737f20-9c32-477b-92c4-741d26b0074c-kube-api-access-659jg\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-kubelet\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-systemd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.702915    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.703389    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.703424    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qpwxd\""
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.703591    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.703597    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 25 00:35:47.703773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.703748    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.706100 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.706079    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.706318 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.706297    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 25 00:35:47.706437 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.706396    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rg2k4\""
Jul 25 00:35:47.706437 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.706418    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 00:35:47.706637 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.706619    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:35:47.708441 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.708402    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 25 00:35:47.708563 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.708516    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4jgts\""
Jul 25 00:35:47.708563 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.708544    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.708670 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.708594    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 25 00:35:47.708670 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.708616    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:47.729760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.729739    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4vgl"
Jul 25 00:35:47.742431 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.742354    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4vgl"
Jul 25 00:35:47.791861 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.791840    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 25 00:35:47.803367 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-multus\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-system-cni-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-dbus\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-lib-modules\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803456    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-multus\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-os-release\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803469    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-system-cni-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-device-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-os-release\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-sys-fs\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-kubelet-config\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803589    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-device-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-kubernetes\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-tmp\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803651    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-kubelet-config\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-sys-fs\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16938946-1f5a-4fa0-af4c-c997a1b62d53-dbus\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803657    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-kubelet\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-kubelet\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-log-socket\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803781    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-env-overrides\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.803852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803829    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-log-socket\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-host\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-socket-dir-parent\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-node-log\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-config\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-netns\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803983    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-socket-dir-parent\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.803994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-tmp-dir\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-netns\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-node-log\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-netns\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovn-node-metrics-cert\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804081    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-netns\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804158    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.804648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-conf-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-multus-certs\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zbcw\" (UniqueName: \"kubernetes.io/projected/59310f30-69f1-4d8b-8939-2320ae3ca7b4-kube-api-access-9zbcw\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-bin\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-env-overrides\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804364    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-multus-certs\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804385    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11348c90-62d8-44b8-b8fe-0f91fc13464d-iptables-alerter-script\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804390    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-conf-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804348    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-tmp-dir\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljzk9\" (UniqueName: \"kubernetes.io/projected/11348c90-62d8-44b8-b8fe-0f91fc13464d-kube-api-access-ljzk9\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804420    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-bin\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804473    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-cni-binary-copy\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-slash\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804587    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-config\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-slash\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804719    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-registration-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804796    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-var-lib-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804833    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11348c90-62d8-44b8-b8fe-0f91fc13464d-host-slash\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysconfig\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804860    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-multus-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-sys\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804910    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-var-lib-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804912    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b24z\" (UniqueName: \"kubernetes.io/projected/c67c6b28-9884-4498-bd1f-0b5f7d52231f-kube-api-access-5b24z\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-hosts-file\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-systemd-units\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-script-lib\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8ppf\" (UniqueName: \"kubernetes.io/projected/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-kube-api-access-r8ppf\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805040    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-hosts-file\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.805938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-system-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.804999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-registration-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805083    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805110    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpsp5\" (UniqueName: \"kubernetes.io/projected/90764f53-efa3-453c-9e7d-0a38b182318b-kube-api-access-jpsp5\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/59310f30-69f1-4d8b-8939-2320ae3ca7b4-serviceca\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805167    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-system-cni-dir\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-etc-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-hostroot\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805215    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805233    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-netd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805083    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-cni-binary-copy\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cnibin\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-conf\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.806707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805373    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-hostroot\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-konnectivity-ca\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-multus-daemon-config\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805429    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-systemd-units\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805438    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-659jg\" (UniqueName: \"kubernetes.io/projected/02737f20-9c32-477b-92c4-741d26b0074c-kube-api-access-659jg\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805448    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovnkube-script-lib\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805481    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cnibin\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805488    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-cni-netd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.805235    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-kubelet\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805524    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-systemd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.805569    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:35:48.305549224 +0000 UTC m=+3.016967606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805629    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-systemd\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-tuned\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.807604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-ovn\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56kgv\" (UniqueName: \"kubernetes.io/projected/0c0b6892-c857-4590-8b2d-ca8b10d81751-kube-api-access-56kgv\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-os-release\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805720    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-etc-openvswitch\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805779    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-kubelet\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805664    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/59310f30-69f1-4d8b-8939-2320ae3ca7b4-serviceca\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-ovn\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805836    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-run-systemd\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-os-release\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805861    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-modprobe-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805907    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-var-lib-kubelet\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-agent-certs\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805947    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90764f53-efa3-453c-9e7d-0a38b182318b-multus-daemon-config\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb4vr\" (UniqueName: \"kubernetes.io/projected/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-kube-api-access-nb4vr\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.805995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-cnibin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-k8s-cni-cncf-io\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-run-k8s-cni-cncf-io\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.808451 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-cnibin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-run\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806167    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-bin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-etc-kubernetes\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59310f30-69f1-4d8b-8939-2320ae3ca7b4-host\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-host-var-lib-cni-bin\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-socket-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806280    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90764f53-efa3-453c-9e7d-0a38b182318b-etc-kubernetes\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2s8\" (UniqueName: \"kubernetes.io/projected/8a98c871-1d3f-484c-80b5-059fd1f301c0-kube-api-access-9t2s8\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806314    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59310f30-69f1-4d8b-8939-2320ae3ca7b4-host\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c0b6892-c857-4590-8b2d-ca8b10d81751-host-run-ovn-kubernetes\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.809177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.806391    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/02737f20-9c32-477b-92c4-741d26b0074c-socket-dir\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.809577 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.809187    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c0b6892-c857-4590-8b2d-ca8b10d81751-ovn-node-metrics-cert\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.812959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.812909    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zbcw\" (UniqueName: \"kubernetes.io/projected/59310f30-69f1-4d8b-8939-2320ae3ca7b4-kube-api-access-9zbcw\") pod \"node-ca-w5qb9\" (UID: \"59310f30-69f1-4d8b-8939-2320ae3ca7b4\") " pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:47.815553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.815482    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb4vr\" (UniqueName: \"kubernetes.io/projected/41a7e68e-c7eb-4150-9b35-424eaf1cb3f7-kube-api-access-nb4vr\") pod \"node-resolver-k5qjn\" (UID: \"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7\") " pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:47.815764 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.815717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56kgv\" (UniqueName: \"kubernetes.io/projected/0c0b6892-c857-4590-8b2d-ca8b10d81751-kube-api-access-56kgv\") pod \"ovnkube-node-84xcm\" (UID: \"0c0b6892-c857-4590-8b2d-ca8b10d81751\") " pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:47.815927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.815906    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-659jg\" (UniqueName: \"kubernetes.io/projected/02737f20-9c32-477b-92c4-741d26b0074c-kube-api-access-659jg\") pod \"aws-ebs-csi-driver-node-9vrkf\" (UID: \"02737f20-9c32-477b-92c4-741d26b0074c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:47.816370 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.816342    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8ppf\" (UniqueName: \"kubernetes.io/projected/b5e52a4f-42bd-4e53-b34c-a17c2b068f12-kube-api-access-r8ppf\") pod \"multus-additional-cni-plugins-j5kdf\" (UID: \"b5e52a4f-42bd-4e53-b34c-a17c2b068f12\") " pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:47.816722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.816705    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpsp5\" (UniqueName: \"kubernetes.io/projected/90764f53-efa3-453c-9e7d-0a38b182318b-kube-api-access-jpsp5\") pod \"multus-g9xqk\" (UID: \"90764f53-efa3-453c-9e7d-0a38b182318b\") " pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.906631 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-conf\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906631 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906630    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-konnectivity-ca\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-systemd\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-tuned\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-modprobe-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-var-lib-kubelet\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906763    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-systemd\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-agent-certs\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906827    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-var-lib-kubelet\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-run\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906838    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-modprobe-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.906868 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2s8\" (UniqueName: \"kubernetes.io/projected/8a98c871-1d3f-484c-80b5-059fd1f301c0-kube-api-access-9t2s8\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906914    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-conf\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-lib-modules\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-run\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.906978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-kubernetes\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-tmp\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907048    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-lib-modules\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-host\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11348c90-62d8-44b8-b8fe-0f91fc13464d-iptables-alerter-script\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljzk9\" (UniqueName: \"kubernetes.io/projected/11348c90-62d8-44b8-b8fe-0f91fc13464d-kube-api-access-ljzk9\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907191    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-kubernetes\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11348c90-62d8-44b8-b8fe-0f91fc13464d-host-slash\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907149    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysctl-d\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.907422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907357    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-konnectivity-ca\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907458    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-host\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907425    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11348c90-62d8-44b8-b8fe-0f91fc13464d-host-slash\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysconfig\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907549    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-sys\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907550    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-sysconfig\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b24z\" (UniqueName: \"kubernetes.io/projected/c67c6b28-9884-4498-bd1f-0b5f7d52231f-kube-api-access-5b24z\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a98c871-1d3f-484c-80b5-059fd1f301c0-sys\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.907715    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.907723    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11348c90-62d8-44b8-b8fe-0f91fc13464d-iptables-alerter-script\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.908315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.907773    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:35:48.407754713 +0000 UTC m=+3.119173118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:47.909977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.909952    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-etc-tuned\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.910089 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.909985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b5b0329-a6a0-4bae-8f69-7cdf270262d2-agent-certs\") pod \"konnectivity-agent-69jv9\" (UID: \"3b5b0329-a6a0-4bae-8f69-7cdf270262d2\") " pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:47.910354 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.910332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a98c871-1d3f-484c-80b5-059fd1f301c0-tmp\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.914625 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.914235    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:35:47.914625 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.914263    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:35:47.914625 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.914277    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:47.914625 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:47.914340    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:35:48.414324131 +0000 UTC m=+3.125742523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:47.916818 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.916727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljzk9\" (UniqueName: \"kubernetes.io/projected/11348c90-62d8-44b8-b8fe-0f91fc13464d-kube-api-access-ljzk9\") pod \"iptables-alerter-jgzpl\" (UID: \"11348c90-62d8-44b8-b8fe-0f91fc13464d\") " pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:47.917064 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.917035    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2s8\" (UniqueName: \"kubernetes.io/projected/8a98c871-1d3f-484c-80b5-059fd1f301c0-kube-api-access-9t2s8\") pod \"tuned-8hswt\" (UID: \"8a98c871-1d3f-484c-80b5-059fd1f301c0\") " pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:47.917562 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.917488    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b24z\" (UniqueName: \"kubernetes.io/projected/c67c6b28-9884-4498-bd1f-0b5f7d52231f-kube-api-access-5b24z\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:47.931686 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.931636    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal" event={"ID":"9b454936a2c3aaec87ddb9b7f44eca43","Type":"ContainerStarted","Data":"c97ebeb5b77fdfe7ad54203ca4cdb2946405de405647d436d1fd5732232d27b5"}
Jul 25 00:35:47.933388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.933364    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal" event={"ID":"b6a7dffde4427b22d8c34d040f4ff0c1","Type":"ContainerStarted","Data":"74eeaca499dd60f89c1e17e23ac2e514b88b417193cd01bd28fae5fc0af7944b"}
Jul 25 00:35:47.958385 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.958356    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 00:35:47.992527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.992486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9xqk"
Jul 25 00:35:47.998099 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:47.998077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5qb9"
Jul 25 00:35:48.004253 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.004222    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90764f53_efa3_453c_9e7d_0a38b182318b.slice/crio-f76c5668ba8c8f45e3bc26ee7b16e012dfa1fa0b2c785dc460cf3a40527ab4a7 WatchSource:0}: Error finding container f76c5668ba8c8f45e3bc26ee7b16e012dfa1fa0b2c785dc460cf3a40527ab4a7: Status 404 returned error can't find the container with id f76c5668ba8c8f45e3bc26ee7b16e012dfa1fa0b2c785dc460cf3a40527ab4a7
Jul 25 00:35:48.009585 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.009564    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf"
Jul 25 00:35:48.009896 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.009876    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59310f30_69f1_4d8b_8939_2320ae3ca7b4.slice/crio-a279760ca4443974abbbe43afd30e719b81b6c1d06f2732c8b2ae78b543d8840 WatchSource:0}: Error finding container a279760ca4443974abbbe43afd30e719b81b6c1d06f2732c8b2ae78b543d8840: Status 404 returned error can't find the container with id a279760ca4443974abbbe43afd30e719b81b6c1d06f2732c8b2ae78b543d8840
Jul 25 00:35:48.015679 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.015652    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5qjn"
Jul 25 00:35:48.019624 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.019596    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02737f20_9c32_477b_92c4_741d26b0074c.slice/crio-58b817a9a516c7fc918de261365b294e650210c8483163222e62e7ea5590b788 WatchSource:0}: Error finding container 58b817a9a516c7fc918de261365b294e650210c8483163222e62e7ea5590b788: Status 404 returned error can't find the container with id 58b817a9a516c7fc918de261365b294e650210c8483163222e62e7ea5590b788
Jul 25 00:35:48.023315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.023291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:35:48.025726 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.025701    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a7e68e_c7eb_4150_9b35_424eaf1cb3f7.slice/crio-6859c7ac56791b1399042fde16b863df4dead8d9091819b3667882361806bde0 WatchSource:0}: Error finding container 6859c7ac56791b1399042fde16b863df4dead8d9091819b3667882361806bde0: Status 404 returned error can't find the container with id 6859c7ac56791b1399042fde16b863df4dead8d9091819b3667882361806bde0
Jul 25 00:35:48.029982 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.029960    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5kdf"
Jul 25 00:35:48.033992 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.033960    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0b6892_c857_4590_8b2d_ca8b10d81751.slice/crio-8a501b7cdeec17bda1fcdbce3ef6f3a63f9c2d4912e59a1365b584de54a22305 WatchSource:0}: Error finding container 8a501b7cdeec17bda1fcdbce3ef6f3a63f9c2d4912e59a1365b584de54a22305: Status 404 returned error can't find the container with id 8a501b7cdeec17bda1fcdbce3ef6f3a63f9c2d4912e59a1365b584de54a22305
Jul 25 00:35:48.038669 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.038649    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jgzpl"
Jul 25 00:35:48.040218 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.040194    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e52a4f_42bd_4e53_b34c_a17c2b068f12.slice/crio-bb77c6bb89fdb3ab42cfc514cd8863de2210c297e76117141167e8ee75434c3a WatchSource:0}: Error finding container bb77c6bb89fdb3ab42cfc514cd8863de2210c297e76117141167e8ee75434c3a: Status 404 returned error can't find the container with id bb77c6bb89fdb3ab42cfc514cd8863de2210c297e76117141167e8ee75434c3a
Jul 25 00:35:48.045783 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.045712    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hswt"
Jul 25 00:35:48.048631 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.048487    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11348c90_62d8_44b8_b8fe_0f91fc13464d.slice/crio-18f65dfe3b2782861b6d576b1ae52895a89737073119876cf4af12f0aef50f68 WatchSource:0}: Error finding container 18f65dfe3b2782861b6d576b1ae52895a89737073119876cf4af12f0aef50f68: Status 404 returned error can't find the container with id 18f65dfe3b2782861b6d576b1ae52895a89737073119876cf4af12f0aef50f68
Jul 25 00:35:48.051427 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.051377    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:35:48.056025 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.056003    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a98c871_1d3f_484c_80b5_059fd1f301c0.slice/crio-5864ecc6f30df06255c070de628bb98795f2398bee90330d214ef22ac4639355 WatchSource:0}: Error finding container 5864ecc6f30df06255c070de628bb98795f2398bee90330d214ef22ac4639355: Status 404 returned error can't find the container with id 5864ecc6f30df06255c070de628bb98795f2398bee90330d214ef22ac4639355
Jul 25 00:35:48.063549 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:35:48.063520    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5b0329_a6a0_4bae_8f69_7cdf270262d2.slice/crio-19a68b0d862e7de8c7b4d5d88f43d7909b4d7066ea56d61ef6969ca726901126 WatchSource:0}: Error finding container 19a68b0d862e7de8c7b4d5d88f43d7909b4d7066ea56d61ef6969ca726901126: Status 404 returned error can't find the container with id 19a68b0d862e7de8c7b4d5d88f43d7909b4d7066ea56d61ef6969ca726901126
Jul 25 00:35:48.310642 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.310520    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:48.310798 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.310694    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:48.310798 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.310760    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:35:49.310741451 +0000 UTC m=+4.022159846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:48.411852 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.411817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:48.412029 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.411976    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:48.412085 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.412050    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:35:49.412033065 +0000 UTC m=+4.123451444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:48.512425 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.512384    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:48.512627 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.512538    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:35:48.512627 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.512577    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:35:48.512627 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.512591    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:48.512875 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.512662    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:35:49.512642193 +0000 UTC m=+4.224060576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:48.746388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.746323    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 00:30:48 +0000 UTC" deadline="2028-04-03 03:22:28.09010869 +0000 UTC"
Jul 25 00:35:48.746388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.746385    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14834h46m39.343727945s"
Jul 25 00:35:48.927401 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.927327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:48.927577 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.927487    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:48.927959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.927938    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:48.928056 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:48.928041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:48.944105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.944073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"8a501b7cdeec17bda1fcdbce3ef6f3a63f9c2d4912e59a1365b584de54a22305"}
Jul 25 00:35:48.953894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.953863    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5qjn" event={"ID":"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7","Type":"ContainerStarted","Data":"6859c7ac56791b1399042fde16b863df4dead8d9091819b3667882361806bde0"}
Jul 25 00:35:48.970544 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.969962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5qb9" event={"ID":"59310f30-69f1-4d8b-8939-2320ae3ca7b4","Type":"ContainerStarted","Data":"a279760ca4443974abbbe43afd30e719b81b6c1d06f2732c8b2ae78b543d8840"}
Jul 25 00:35:48.975189 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.975135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9xqk" event={"ID":"90764f53-efa3-453c-9e7d-0a38b182318b","Type":"ContainerStarted","Data":"f76c5668ba8c8f45e3bc26ee7b16e012dfa1fa0b2c785dc460cf3a40527ab4a7"}
Jul 25 00:35:48.988857 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.988829    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hswt" event={"ID":"8a98c871-1d3f-484c-80b5-059fd1f301c0","Type":"ContainerStarted","Data":"5864ecc6f30df06255c070de628bb98795f2398bee90330d214ef22ac4639355"}
Jul 25 00:35:48.992428 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.992402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jgzpl" event={"ID":"11348c90-62d8-44b8-b8fe-0f91fc13464d","Type":"ContainerStarted","Data":"18f65dfe3b2782861b6d576b1ae52895a89737073119876cf4af12f0aef50f68"}
Jul 25 00:35:48.999666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:48.999639    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerStarted","Data":"bb77c6bb89fdb3ab42cfc514cd8863de2210c297e76117141167e8ee75434c3a"}
Jul 25 00:35:49.002457 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.002431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf" event={"ID":"02737f20-9c32-477b-92c4-741d26b0074c","Type":"ContainerStarted","Data":"58b817a9a516c7fc918de261365b294e650210c8483163222e62e7ea5590b788"}
Jul 25 00:35:49.008020 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.007996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-69jv9" event={"ID":"3b5b0329-a6a0-4bae-8f69-7cdf270262d2","Type":"ContainerStarted","Data":"19a68b0d862e7de8c7b4d5d88f43d7909b4d7066ea56d61ef6969ca726901126"}
Jul 25 00:35:49.320105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.319966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:49.320287 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.320162    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:49.320287 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.320248    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:35:51.320224931 +0000 UTC m=+6.031643326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:49.421953 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.421323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:49.421953 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.421512    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:49.421953 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.421605    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:35:51.421584959 +0000 UTC m=+6.133003358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:49.522428 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.522389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:49.522650 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.522627    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:35:49.522750 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.522661    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:35:49.522750 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.522675    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:49.522750 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.522738    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:35:51.52271817 +0000 UTC m=+6.234136568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:49.746729 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.746620    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 00:30:48 +0000 UTC" deadline="2028-01-26 00:35:04.268872206 +0000 UTC"
Jul 25 00:35:49.746729 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.746664    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13199h59m14.52221341s"
Jul 25 00:35:49.931210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:49.930328    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:49.931210 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:49.930732    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:50.036780 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:50.036689    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal" event={"ID":"b6a7dffde4427b22d8c34d040f4ff0c1","Type":"ContainerStarted","Data":"fd880fc4ebeeff60bfeca2551673d01573c590e886c679c47c24220a94516b56"}
Jul 25 00:35:50.927075 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:50.926987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:50.927556 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:50.927138    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:50.927649 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:50.927626    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:50.927750 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:50.927725    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:51.047955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.046959    2574 generic.go:356] "Generic (PLEG): container finished" podID="b6a7dffde4427b22d8c34d040f4ff0c1" containerID="fd880fc4ebeeff60bfeca2551673d01573c590e886c679c47c24220a94516b56" exitCode=0
Jul 25 00:35:51.047955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.047024    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal" event={"ID":"b6a7dffde4427b22d8c34d040f4ff0c1","Type":"ContainerDied","Data":"fd880fc4ebeeff60bfeca2551673d01573c590e886c679c47c24220a94516b56"}
Jul 25 00:35:51.337112 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.337026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:51.337285 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.337222    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:51.337350 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.337291    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:35:55.337271041 +0000 UTC m=+10.048689435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:51.437958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.437914    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:51.438189 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.438162    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:51.438281 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.438250    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:35:55.438226537 +0000 UTC m=+10.149644919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:51.538961 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.538843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:51.539131 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.539055    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:35:51.539131 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.539089    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:35:51.539131 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.539104    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:51.539322 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.539168    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:35:55.539146995 +0000 UTC m=+10.250565389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:51.926915 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:51.926879    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:51.927119 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:51.927039    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:52.927490 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:52.926742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:52.927490 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:52.926915    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:52.927490 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:52.927328    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:52.927490 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:52.927448    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:53.926655 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:53.926493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:53.926829 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:53.926673    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:54.927113 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:54.927077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:54.927113 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:54.927105    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:54.927729 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:54.927246    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:54.927729 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:54.927479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:55.373333 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:55.372619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:55.373333 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.372850    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:55.373333 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.372917    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:03.372897699 +0000 UTC m=+18.084316081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:35:55.473647 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:55.473614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:55.473835 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.473797    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:55.473911 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.473872    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:36:03.473850333 +0000 UTC m=+18.185268730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:35:55.574540 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:55.574429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:55.579549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.577502    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:35:55.579549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.577558    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:35:55.579549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.577574    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:55.579549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.577651    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:36:03.577631409 +0000 UTC m=+18.289049807 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:35:55.928675 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:55.928153    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:55.928675 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:55.928287    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:56.927086 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:56.926990    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:56.927289 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:56.927141    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:56.927642 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:56.927441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:56.927642 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:56.927600    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:57.926614 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:57.926579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:57.927135 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:57.926744    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:35:58.927105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:58.927067    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:35:58.927618 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:58.927067    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:35:58.927618 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:58.927190    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:35:58.927618 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:58.927314    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:35:59.926618 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:35:59.926583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:35:59.926758 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:35:59.926711    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:00.926881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:00.926838    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:00.927337 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:00.926975    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:00.927337 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:00.927034    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:00.927337 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:00.927131    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:01.926249 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:01.926218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:01.926421 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:01.926354    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:02.926835 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:02.926799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:02.927304 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:02.926937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:02.927304 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:02.926999    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:02.927304 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:02.927123    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:03.437335 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:03.437294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:03.437497 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.437413    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:03.437497 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.437485    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:19.437466338 +0000 UTC m=+34.148884731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:03.538646 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:03.538614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:03.538796 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.538766    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:03.538838 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.538827    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:36:19.538808036 +0000 UTC m=+34.250226412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:03.639637 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:03.639605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:03.639801 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.639780    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:03.639862 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.639808    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:03.639862 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.639818    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:03.639930 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.639867    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:36:19.639853395 +0000 UTC m=+34.351271772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:03.926901 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:03.926873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:03.927308 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:03.927004    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:04.926756 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:04.926712    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:04.926912 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:04.926858    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:04.926979 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:04.926909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:04.927304 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:04.927007    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:05.927032 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:05.927002    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:05.927472 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:05.927117    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:06.927138 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:06.927103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:06.927549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:06.927217    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:06.927549 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:06.927267    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:06.927549 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:06.927378    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:07.926330 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:07.926301    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:07.926609 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:07.926428    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:08.927003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:08.926667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:08.927734 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:08.926723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:08.927734 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:08.927089    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:08.927734 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:08.927194    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:09.125569 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.125534    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="cdb5e68cb092997561348635bf24969ebb23d17fad24c073dc84af83d391bfea" exitCode=0
Jul 25 00:36:09.125764 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.125622    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"cdb5e68cb092997561348635bf24969ebb23d17fad24c073dc84af83d391bfea"}
Jul 25 00:36:09.127464 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.127436    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf" event={"ID":"02737f20-9c32-477b-92c4-741d26b0074c","Type":"ContainerStarted","Data":"037caf987c47956a6c9af00a9786cbb16005aff3f3ca3c4c96de242b01173188"}
Jul 25 00:36:09.129466 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.129444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal" event={"ID":"b6a7dffde4427b22d8c34d040f4ff0c1","Type":"ContainerStarted","Data":"1e4aef03998c5b3f2dfa8872606810a21392b68311b2956c2511c90ca4ed1eec"}
Jul 25 00:36:09.130955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.130935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-69jv9" event={"ID":"3b5b0329-a6a0-4bae-8f69-7cdf270262d2","Type":"ContainerStarted","Data":"a9a5c1c62193708ea79728df304443e9e3a1eaa7e34225854528eaba3a225a1e"}
Jul 25 00:36:09.137426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137396    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"a2254e91d6f45a5556ed10539b4a7d384c5eec98f0a850ec0d31a42bf465d7d5"}
Jul 25 00:36:09.137539 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137502    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"a22b84d4a42640fc953b215d5ff0ef44a20126da04c21810908ab44f01c37f0a"}
Jul 25 00:36:09.137587 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"d8b14b5379ac423126502d3d7a58de47b00a7351de3a44deb20d4e6f4c5d2099"}
Jul 25 00:36:09.137625 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137615    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"4df6f4d74cd6c1b537a506d10063b3d597a6d2b5f69027fe0c4ebb690f77f0f4"}
Jul 25 00:36:09.137670 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"1af3d8e92c919af0388ac93ea13fe130f4f019ab871ebb939cbc760c00cb804c"}
Jul 25 00:36:09.137720 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.137684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"65058993cb23d96961cdabc0c5ca60acbc820c99cb3be73641e4a96d49db0d61"}
Jul 25 00:36:09.139018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.138993    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5qjn" event={"ID":"41a7e68e-c7eb-4150-9b35-424eaf1cb3f7","Type":"ContainerStarted","Data":"c6a84e4b181b6280853aa157d87844c01def0980be8c147b356e19b37830bd1d"}
Jul 25 00:36:09.140502 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.140475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5qb9" event={"ID":"59310f30-69f1-4d8b-8939-2320ae3ca7b4","Type":"ContainerStarted","Data":"e21dc0c5b49f02e72523560e8def9e045d2ce90e2f62f48498c0d78dd8724bad"}
Jul 25 00:36:09.142132 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.142111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9xqk" event={"ID":"90764f53-efa3-453c-9e7d-0a38b182318b","Type":"ContainerStarted","Data":"c573c14d9a8e6808be04138392aa6f28b2919396c3302d209d6310aabc097ac7"}
Jul 25 00:36:09.143895 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.143875    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal" event={"ID":"9b454936a2c3aaec87ddb9b7f44eca43","Type":"ContainerStarted","Data":"db7bb0fbdb9af98467c973db0392b3d9c418e913036986b67ec99fbd11affa43"}
Jul 25 00:36:09.145332 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.145310    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hswt" event={"ID":"8a98c871-1d3f-484c-80b5-059fd1f301c0","Type":"ContainerStarted","Data":"6f6cdd16c10c029dab1d28f5993c93f94f1e4b55728be2dc8883aa20a4ccae97"}
Jul 25 00:36:09.166527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.166461    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8hswt" podStartSLOduration=2.210843745 podStartE2EDuration="22.166446231s" totalImagesPullingTime="19.955602486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.1662014 +0000 UTC m=+23.877619796" watchObservedRunningTime="2026-07-25 00:36:09.166446231 +0000 UTC m=+23.877864630"
Jul 25 00:36:09.181705 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.181652    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-233.ec2.internal" podStartSLOduration=21.181633219 podStartE2EDuration="21.181633219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.18138545 +0000 UTC m=+23.892803850" watchObservedRunningTime="2026-07-25 00:36:09.181633219 +0000 UTC m=+23.893051619"
Jul 25 00:36:09.222720 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.222696    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:36:09.223202 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.223185    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:36:09.227271 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.227235    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-233.ec2.internal" podStartSLOduration=21.227222247 podStartE2EDuration="21.227222247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.205237687 +0000 UTC m=+23.916656086" watchObservedRunningTime="2026-07-25 00:36:09.227222247 +0000 UTC m=+23.938640646"
Jul 25 00:36:09.242717 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.242676    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g9xqk" podStartSLOduration=2.187292044 podStartE2EDuration="22.242661805s" totalImagesPullingTime="20.055369761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.226980132 +0000 UTC m=+23.938398543" watchObservedRunningTime="2026-07-25 00:36:09.242661805 +0000 UTC m=+23.954080204"
Jul 25 00:36:09.256880 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.256838    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-69jv9" podStartSLOduration=2.31901821 podStartE2EDuration="22.256824548s" totalImagesPullingTime="19.937806338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.256437652 +0000 UTC m=+23.967856051" watchObservedRunningTime="2026-07-25 00:36:09.256824548 +0000 UTC m=+23.968242947"
Jul 25 00:36:09.256992 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.256975    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k5qjn" podStartSLOduration=2.280495295 podStartE2EDuration="22.256970943s" totalImagesPullingTime="19.976475648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.242571424 +0000 UTC m=+23.953989823" watchObservedRunningTime="2026-07-25 00:36:09.256970943 +0000 UTC m=+23.968389341"
Jul 25 00:36:09.271823 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.271782    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w5qb9" podStartSLOduration=2.280125922 podStartE2EDuration="22.271768907s" totalImagesPullingTime="19.991642985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:09.27144226 +0000 UTC m=+23.982860660" watchObservedRunningTime="2026-07-25 00:36:09.271768907 +0000 UTC m=+23.983187307"
Jul 25 00:36:09.495993 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.495965    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 25 00:36:09.918448 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.918330    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-25T00:36:09.49598732Z","UUID":"484dd1e6-ade2-48cf-896a-55355d4bde9d","Handler":null,"Name":"","Endpoint":""}
Jul 25 00:36:09.920315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.920285    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 25 00:36:09.920463 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.920330    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 25 00:36:09.926223 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:09.926202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:09.926353 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:09.926329    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:10.149873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.149831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jgzpl" event={"ID":"11348c90-62d8-44b8-b8fe-0f91fc13464d","Type":"ContainerStarted","Data":"b7b708037517767595ac4e7cfef24c459d00553b52366af7dfafa02b6312474d"}
Jul 25 00:36:10.152076 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.152044    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf" event={"ID":"02737f20-9c32-477b-92c4-741d26b0074c","Type":"ContainerStarted","Data":"e6152e2cbe4cae5d161ef5143ba25a1dd6291d8b4cb8f7da24f95fac4a39134a"}
Jul 25 00:36:10.152415 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.152397    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:36:10.152822 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.152803    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-69jv9"
Jul 25 00:36:10.165563 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.165499    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jgzpl" podStartSLOduration=3.213556884 podStartE2EDuration="23.16548075s" totalImagesPullingTime="19.951923866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:10.164918026 +0000 UTC m=+24.876336427" watchObservedRunningTime="2026-07-25 00:36:10.16548075 +0000 UTC m=+24.876899150"
Jul 25 00:36:10.926495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.926437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:10.926644 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:10.926445    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:10.926644 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:10.926588    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:10.926644 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:10.926624    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:11.157115 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:11.157069    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"8b2372709d70c7988b55fd90768ece8ba75abda910b092336684efb2e47af2f1"}
Jul 25 00:36:11.159263 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:11.159228    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf" event={"ID":"02737f20-9c32-477b-92c4-741d26b0074c","Type":"ContainerStarted","Data":"b64d4caf633a3176809dd1fc8da56cf1a5a9500d0b56c1a0f4ba4662f8b03472"}
Jul 25 00:36:11.179856 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:11.179761    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vrkf" podStartSLOduration=1.857507611 podStartE2EDuration="24.179739841s" totalImagesPullingTime="22.32223223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:11.179330251 +0000 UTC m=+25.890748650" watchObservedRunningTime="2026-07-25 00:36:11.179739841 +0000 UTC m=+25.891158242"
Jul 25 00:36:11.926502 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:11.926314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:11.926674 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:11.926625    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:12.926611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:12.926572    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:12.927121 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:12.926572    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:12.927121 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:12.926699    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:12.927121 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:12.926808    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:13.927152 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:13.926967    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:13.927990 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:13.927255    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:14.169791 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.169756    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" event={"ID":"0c0b6892-c857-4590-8b2d-ca8b10d81751","Type":"ContainerStarted","Data":"7c29d0db292509a14978dcdf5b691295c034e4f50e173dee238ab0437ef64c05"}
Jul 25 00:36:14.170031 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.169983    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:14.170105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.170058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:14.174572 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.174545    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="d5f6640eb0583ffcd9d333051dd1af9eb0c92b2f1d4e071315c59324e4906a6c" exitCode=0
Jul 25 00:36:14.174692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.174579    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"d5f6640eb0583ffcd9d333051dd1af9eb0c92b2f1d4e071315c59324e4906a6c"}
Jul 25 00:36:14.186223 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.186204    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:14.197698 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.197656    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm" podStartSLOduration=7.147353246 podStartE2EDuration="27.197642654s" totalImagesPullingTime="20.050289408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:14.19684611 +0000 UTC m=+28.908264510" watchObservedRunningTime="2026-07-25 00:36:14.197642654 +0000 UTC m=+28.909061052"
Jul 25 00:36:14.926716 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.926683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:14.926919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:14.926683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:14.926919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:14.926796    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:14.926919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:14.926893    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:15.177283 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.177109    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:15.193141 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.193109    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:15.451756 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.451664    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wf7xm"]
Jul 25 00:36:15.451900 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.451781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:15.451945 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:15.451891    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:15.454390 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.454360    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t8lqb"]
Jul 25 00:36:15.454532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.454449    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:15.454610 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:15.454552    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:15.457320 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.457296    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdr58"]
Jul 25 00:36:15.457441 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:15.457386    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:15.457496 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:15.457462    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:16.180829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:16.180793    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="dd90c9b5943b5b2f83766065ca1728b24e710d8be9645dcae0f02dcbb4c223f5" exitCode=0
Jul 25 00:36:16.181304 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:16.180871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"dd90c9b5943b5b2f83766065ca1728b24e710d8be9645dcae0f02dcbb4c223f5"}
Jul 25 00:36:16.926452 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:16.926415    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:16.926649 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:16.926461    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:16.926649 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:16.926527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:16.926649 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:16.926542    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:16.926649 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:16.926607    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:16.926807 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:16.926664    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:18.187572 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:18.187366    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="4a6807a37c46e3c635780c6ea57b7701427634340e65c8748bfd64ccd039c16b" exitCode=0
Jul 25 00:36:18.188007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:18.187411    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"4a6807a37c46e3c635780c6ea57b7701427634340e65c8748bfd64ccd039c16b"}
Jul 25 00:36:18.926426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:18.926379    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:18.926635 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:18.926506    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:18.926635 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:18.926527    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7xm" podUID="16938946-1f5a-4fa0-af4c-c997a1b62d53"
Jul 25 00:36:18.926635 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:18.926537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:18.926795 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:18.926637    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:36:18.926795 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:18.926714    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t8lqb" podUID="77288b90-5cb2-4722-a2b3-aa1ecfb750bc"
Jul 25 00:36:19.453857 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:19.453820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:19.454271 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.453941    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:19.454271 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.454016    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret podName:16938946-1f5a-4fa0-af4c-c997a1b62d53 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.45399646 +0000 UTC m=+66.165414841 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret") pod "global-pull-secret-syncer-wf7xm" (UID: "16938946-1f5a-4fa0-af4c-c997a1b62d53") : object "kube-system"/"original-pull-secret" not registered
Jul 25 00:36:19.555177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:19.555137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:19.555360 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.555321    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:19.555417 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.555398    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.555378572 +0000 UTC m=+66.266796954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 00:36:19.656139 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:19.656107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:19.656342 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.656310    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 00:36:19.656342 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.656341    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 00:36:19.656483 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.656355    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fmm59 for pod openshift-network-diagnostics/network-check-target-t8lqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:19.656483 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:19.656425    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59 podName:77288b90-5cb2-4722-a2b3-aa1ecfb750bc nodeName:}" failed. No retries permitted until 2026-07-25 00:36:51.656405921 +0000 UTC m=+66.367824299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmm59" (UniqueName: "kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59") pod "network-check-target-t8lqb" (UID: "77288b90-5cb2-4722-a2b3-aa1ecfb750bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 00:36:20.602624 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.602597    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-233.ec2.internal" event="NodeReady"
Jul 25 00:36:20.603132 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.602742    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 25 00:36:20.636495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.636466    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"]
Jul 25 00:36:20.639959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.639939    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.641985 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.641956    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:36:20.642825 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.642807    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 25 00:36:20.642962 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.642940    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d68th\""
Jul 25 00:36:20.643238 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.643223    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 25 00:36:20.645966 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.645949    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.648192 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.648172    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 25 00:36:20.648458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.648427    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 25 00:36:20.648560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.648471    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 25 00:36:20.648812 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.648793    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5x988\""
Jul 25 00:36:20.651087 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.651031    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"]
Jul 25 00:36:20.651893 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.651869    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vvwv5"]
Jul 25 00:36:20.654258 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.654034    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 25 00:36:20.656178 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.656161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.658027 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.657967    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tzzs5"]
Jul 25 00:36:20.658862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.658838    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 25 00:36:20.658958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.658897    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 25 00:36:20.658958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.658915    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b2m9l\""
Jul 25 00:36:20.661812 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.661792    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:36:20.661955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.661940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.666419 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.666401    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 25 00:36:20.666605 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.666547    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 25 00:36:20.666714 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.666697    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 25 00:36:20.667028 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.667010    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vl845\""
Jul 25 00:36:20.668912 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.668891    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvwv5"]
Jul 25 00:36:20.669394 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.669371    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzzs5"]
Jul 25 00:36:20.762455 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjbp\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.762659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.762659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpcxv\" (UniqueName: \"kubernetes.io/projected/8efb9a7a-78b0-4351-8516-1af11cbe34a5-kube-api-access-fpcxv\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.762659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2r97\" (UniqueName: \"kubernetes.io/projected/128b0291-5428-46d8-b2dd-fa8aeb05fe30-kube-api-access-p2r97\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.762659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efb9a7a-78b0-4351-8516-1af11cbe34a5-tmp-dir\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762744    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8efb9a7a-78b0-4351-8516-1af11cbe34a5-config-volume\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.762881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.763148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.763148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.762950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.763148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.763024    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128b0291-5428-46d8-b2dd-fa8aeb05fe30-cert\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.763148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.763060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0fb6c2c8-80c7-467a-86b5-062613ab1f15-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.796727 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.796650    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:20.863611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.863611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpcxv\" (UniqueName: \"kubernetes.io/projected/8efb9a7a-78b0-4351-8516-1af11cbe34a5-kube-api-access-fpcxv\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.863829 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.863741    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:20.863829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2r97\" (UniqueName: \"kubernetes.io/projected/128b0291-5428-46d8-b2dd-fa8aeb05fe30-kube-api-access-p2r97\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.863829 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.863825    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.36380403 +0000 UTC m=+36.075222423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:20.863959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.863959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efb9a7a-78b0-4351-8516-1af11cbe34a5-tmp-dir\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.863959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.863959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863944    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8efb9a7a-78b0-4351-8516-1af11cbe34a5-config-volume\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.863967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864118    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864148    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128b0291-5428-46d8-b2dd-fa8aeb05fe30-cert\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.864482 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0fb6c2c8-80c7-467a-86b5-062613ab1f15-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.864482 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864216    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjbp\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.864482 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.864228    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:20.864482 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.864290    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.36427324 +0000 UTC m=+36.075691633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:20.864482 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8efb9a7a-78b0-4351-8516-1af11cbe34a5-tmp-dir\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.864755 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864620    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8efb9a7a-78b0-4351-8516-1af11cbe34a5-config-volume\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.864927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.864901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.865006 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.864943    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:20.865006 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.864959    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:20.865100 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:20.865013    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:21.364992723 +0000 UTC m=+36.076411135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:20.865100 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.865049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.865421 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.865402    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0fb6c2c8-80c7-467a-86b5-062613ab1f15-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:20.865867 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.865837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.869156 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.869131    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.869156 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.869148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128b0291-5428-46d8-b2dd-fa8aeb05fe30-cert\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.869318 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.869134    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.872840 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.872799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.872973 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.872949    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2r97\" (UniqueName: \"kubernetes.io/projected/128b0291-5428-46d8-b2dd-fa8aeb05fe30-kube-api-access-p2r97\") pod \"ingress-canary-tzzs5\" (UID: \"128b0291-5428-46d8-b2dd-fa8aeb05fe30\") " pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:20.873230 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.873214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpcxv\" (UniqueName: \"kubernetes.io/projected/8efb9a7a-78b0-4351-8516-1af11cbe34a5-kube-api-access-fpcxv\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:20.873440 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.873420    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjbp\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:20.885057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.885029    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:20.926890 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.926855    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:20.926890 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.926891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:20.927128 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.926896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:20.929696 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.929669    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 00:36:20.929696 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.929694    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kt6t\""
Jul 25 00:36:20.929864 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.929698    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7xdtb\""
Jul 25 00:36:20.929864 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.929829    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 00:36:20.930003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.929984    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 00:36:20.930082 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.930047    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 00:36:20.977682 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:20.977651    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzzs5"
Jul 25 00:36:21.126367 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:21.126156    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzzs5"]
Jul 25 00:36:21.368871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:21.368767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:21.368871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:21.368843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:21.368874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.368940    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.368976    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.368988    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.368990    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.369025    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:22.369003689 +0000 UTC m=+37.080422086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.369045    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:22.369035395 +0000 UTC m=+37.080453776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:21.369103 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:21.369061    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:22.3690518 +0000 UTC m=+37.080470178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:22.377036 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:22.376995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:22.377084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:22.377150    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377169    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377189    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377239    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377252    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:24.377231384 +0000 UTC m=+39.088649764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377286    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:24.377274389 +0000 UTC m=+39.088692766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377239    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:22.377767 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:22.377342    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:24.377327639 +0000 UTC m=+39.088746020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:22.553000 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:36:22.552968    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128b0291_5428_46d8_b2dd_fa8aeb05fe30.slice/crio-184490f29a4156e104748f9f1fd50481fc4af4fb504e6ff67ffd15201b3e4ff6 WatchSource:0}: Error finding container 184490f29a4156e104748f9f1fd50481fc4af4fb504e6ff67ffd15201b3e4ff6: Status 404 returned error can't find the container with id 184490f29a4156e104748f9f1fd50481fc4af4fb504e6ff67ffd15201b3e4ff6
Jul 25 00:36:22.870829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:22.870792    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:22.969496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:22.969455    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:23.203316 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:23.203280    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="7bf01ca4469127e30d84f80c1b6f5cb6578b47f36010881dd70722d25471db23" exitCode=0
Jul 25 00:36:23.203494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:23.203364    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"7bf01ca4469127e30d84f80c1b6f5cb6578b47f36010881dd70722d25471db23"}
Jul 25 00:36:23.204654 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:23.204629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzzs5" event={"ID":"128b0291-5428-46d8-b2dd-fa8aeb05fe30","Type":"ContainerStarted","Data":"184490f29a4156e104748f9f1fd50481fc4af4fb504e6ff67ffd15201b3e4ff6"}
Jul 25 00:36:23.840102 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:23.840060    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:23.940662 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:23.940633    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:24.212798 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:24.212767    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5e52a4f-42bd-4e53-b34c-a17c2b068f12" containerID="98e73a2f87b179281528151797363425e032b57464fbc70331360be8fa5de0a2" exitCode=0
Jul 25 00:36:24.212941 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:24.212814    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerDied","Data":"98e73a2f87b179281528151797363425e032b57464fbc70331360be8fa5de0a2"}
Jul 25 00:36:24.395629 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:24.395586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:24.395793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:24.395694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:24.395793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:24.395749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:24.395793 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395785    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:24.395937 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395861    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:28.395835254 +0000 UTC m=+43.107253654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:24.395937 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395869    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:24.395937 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395912    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:28.395901544 +0000 UTC m=+43.107319928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:24.395937 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395927    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:24.396123 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395950    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:24.396123 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:24.395996    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:28.39598115 +0000 UTC m=+43.107399528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:25.218420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.218236    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" event={"ID":"b5e52a4f-42bd-4e53-b34c-a17c2b068f12","Type":"ContainerStarted","Data":"a47d7f55ecffd32c4332c2ca225ce516f97605ea3708b53fdb8e96503052e5a5"}
Jul 25 00:36:25.241952 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.241896    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j5kdf" podStartSLOduration=7.326239608 podStartE2EDuration="38.241881264s" totalImagesPullingTime="30.915641656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:25.240477857 +0000 UTC m=+39.951896255" watchObservedRunningTime="2026-07-25 00:36:25.241881264 +0000 UTC m=+39.953299663"
Jul 25 00:36:25.367986 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.367956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:25.468872 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.468781    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:25.534686 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.534650    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:25.803638 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:25.803551    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:26.222227 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:26.222186    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzzs5" event={"ID":"128b0291-5428-46d8-b2dd-fa8aeb05fe30","Type":"ContainerStarted","Data":"5c3ffa25349e49f178b39f1735e9b226bcf4b2d37af4858a6d7da34d9513ce52"}
Jul 25 00:36:26.237894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:26.237852    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tzzs5" podStartSLOduration=3.521740721 podStartE2EDuration="6.237837811s" totalImagesPullingTime="2.71609709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:26.236999712 +0000 UTC m=+40.948418111" watchObservedRunningTime="2026-07-25 00:36:26.237837811 +0000 UTC m=+40.949256203"
Jul 25 00:36:28.429483 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:28.429441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:28.429527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:28.429602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429626    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429645    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429690    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429694    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429712    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:36.429692122 +0000 UTC m=+51.141110520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429730    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:36.429721616 +0000 UTC m=+51.141139997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:28.429941 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:28.429760    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:36.429741306 +0000 UTC m=+51.141159701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:35.003229 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:35.003191    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:35.175453 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:35.175419    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:35.278210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:35.278133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:36.489476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:36.489441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:36.489506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:36.489547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489589    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489639    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489650    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489663    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:52.489647361 +0000 UTC m=+67.201065738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489687    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:52.489674611 +0000 UTC m=+67.201092988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489700    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:36.489919 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:36.489776    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:36:52.489758426 +0000 UTC m=+67.201176820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:39.597406 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:39.597361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:39.699114 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:39.699075    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:39.942044 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:39.941960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:40.043783 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:40.043750    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:40.456993 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:40.456935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:40.519864 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:40.519833    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:41.416361 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:41.416323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:41.517361 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:41.517329    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:42.366413 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:42.366378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:47.198778 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:47.198746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84xcm"
Jul 25 00:36:47.342091 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:47.342040    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:47.466491 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:47.466460    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:51.497267 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.497226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:51.502926 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.502905    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 00:36:51.511460 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.511440    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16938946-1f5a-4fa0-af4c-c997a1b62d53-original-pull-secret\") pod \"global-pull-secret-syncer-wf7xm\" (UID: \"16938946-1f5a-4fa0-af4c-c997a1b62d53\") " pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:51.538968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.538937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7xm"
Jul 25 00:36:51.600464 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.599919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:36:51.605053 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.605024    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 00:36:51.613922 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:51.613568    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 25 00:36:51.613922 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:51.613659    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:37:55.613634543 +0000 UTC m=+130.325052937 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : secret "metrics-daemon-secret" not found
Jul 25 00:36:51.674354 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.674305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wf7xm"]
Jul 25 00:36:51.678298 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:36:51.678272    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16938946_1f5a_4fa0_af4c_c997a1b62d53.slice/crio-56a7aceefd78cd70dc93132f9e5cf124899a190c27bc2cb8af78ff59592e37d5 WatchSource:0}: Error finding container 56a7aceefd78cd70dc93132f9e5cf124899a190c27bc2cb8af78ff59592e37d5: Status 404 returned error can't find the container with id 56a7aceefd78cd70dc93132f9e5cf124899a190c27bc2cb8af78ff59592e37d5
Jul 25 00:36:51.701100 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.701078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:51.703481 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.703463    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 00:36:51.715601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.715585    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 00:36:51.723844 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.723825    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmm59\" (UniqueName: \"kubernetes.io/projected/77288b90-5cb2-4722-a2b3-aa1ecfb750bc-kube-api-access-fmm59\") pod \"network-check-target-t8lqb\" (UID: \"77288b90-5cb2-4722-a2b3-aa1ecfb750bc\") " pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:51.849752 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.849680    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9kt6t\""
Jul 25 00:36:51.857819 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.857799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:51.974932 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:51.974899    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t8lqb"]
Jul 25 00:36:51.977995 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:36:51.977970    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77288b90_5cb2_4722_a2b3_aa1ecfb750bc.slice/crio-546b03da39fdb9cc0a00ac521c0120d469cac4dca9808c9a3c0b413d77cee679 WatchSource:0}: Error finding container 546b03da39fdb9cc0a00ac521c0120d469cac4dca9808c9a3c0b413d77cee679: Status 404 returned error can't find the container with id 546b03da39fdb9cc0a00ac521c0120d469cac4dca9808c9a3c0b413d77cee679
Jul 25 00:36:52.283942 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:52.283906    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wf7xm" event={"ID":"16938946-1f5a-4fa0-af4c-c997a1b62d53","Type":"ContainerStarted","Data":"56a7aceefd78cd70dc93132f9e5cf124899a190c27bc2cb8af78ff59592e37d5"}
Jul 25 00:36:52.284887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:52.284864    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t8lqb" event={"ID":"77288b90-5cb2-4722-a2b3-aa1ecfb750bc","Type":"ContainerStarted","Data":"546b03da39fdb9cc0a00ac521c0120d469cac4dca9808c9a3c0b413d77cee679"}
Jul 25 00:36:52.506829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:52.506674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:36:52.506829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:52.506727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:36:52.506829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:52.506763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:36:52.506829 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506819    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506899    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506921    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506902    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:24.506878843 +0000 UTC m=+99.218297240 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506967    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.506982    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:24.506964691 +0000 UTC m=+99.218383075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:36:52.507455 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:36:52.507027    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:37:24.507011807 +0000 UTC m=+99.218430197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:36:53.137933 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:53.137842    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:53.238242 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:53.238208    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:56.475663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:56.475624    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:56.544186 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:56.544142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:57.078814 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.078776    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:57.141837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.141809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:36:57.300567 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.300531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wf7xm" event={"ID":"16938946-1f5a-4fa0-af4c-c997a1b62d53","Type":"ContainerStarted","Data":"ab8872632068dbb25932c168a589df89efbfdd5aaaaeea5609d713d82c899368"}
Jul 25 00:36:57.302001 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.301979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t8lqb" event={"ID":"77288b90-5cb2-4722-a2b3-aa1ecfb750bc","Type":"ContainerStarted","Data":"4831bcc085448fba5c658ab437cb9d051a57d175aef8cb62e3e3e5c2979c893a"}
Jul 25 00:36:57.302113 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.302097    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:36:57.316464 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.316422    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wf7xm" podStartSLOduration=65.327033613 podStartE2EDuration="1m10.316409584s" totalImagesPullingTime="4.989375971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:57.315949663 +0000 UTC m=+72.027368061" watchObservedRunningTime="2026-07-25 00:36:57.316409584 +0000 UTC m=+72.027827982"
Jul 25 00:36:57.332719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:36:57.332612    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t8lqb" podStartSLOduration=65.647162127 podStartE2EDuration="1m10.332595098s" totalImagesPullingTime="4.685432971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:36:57.332179141 +0000 UTC m=+72.043597540" watchObservedRunningTime="2026-07-25 00:36:57.332595098 +0000 UTC m=+72.044013496"
Jul 25 00:37:00.925177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:00.925139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:00.984344 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:00.984307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:18.459194 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:18.459060    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:18.527412 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:18.527382    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:24.530034 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:24.529982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:37:24.530034 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:24.530046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:24.530088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530136    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530159    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530178    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530179    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530224    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:38:28.530207337 +0000 UTC m=+163.241625726 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530242    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:38:28.530230851 +0000 UTC m=+163.241649228 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:37:24.530479 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:24.530253    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:38:28.530247889 +0000 UTC m=+163.241666266 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:37:28.308466 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:28.308437    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t8lqb"
Jul 25 00:37:48.959781 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:48.959738    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:49.018989 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:49.018954    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:37:55.652225 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:37:55.652164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:37:55.652650 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:55.652314    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 25 00:37:55.652650 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:37:55.652417    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:39:57.652400646 +0000 UTC m=+252.363819023 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : secret "metrics-daemon-secret" not found
Jul 25 00:38:23.654028 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:23.653983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9" podUID="0fb6c2c8-80c7-467a-86b5-062613ab1f15"
Jul 25 00:38:23.661163 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:23.661133    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"
Jul 25 00:38:23.672465 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:23.672434    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vvwv5" podUID="8efb9a7a-78b0-4351-8516-1af11cbe34a5"
Jul 25 00:38:23.952745 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:23.952651    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:38:24.498215 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:24.498185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:38:24.498416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:24.498185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:38:24.498416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:24.498188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:38:28.576401 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:28.576362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:28.576409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:28.576456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576545    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576578    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576603    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576624    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c5f665fb-xk66n: secret "image-registry-tls" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576633    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert podName:0fb6c2c8-80c7-467a-86b5-062613ab1f15 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:30.576612104 +0000 UTC m=+285.288030503 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-xv9g9" (UID: "0fb6c2c8-80c7-467a-86b5-062613ab1f15") : secret "networking-console-plugin-cert" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576648    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls podName:8efb9a7a-78b0-4351-8516-1af11cbe34a5 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:30.576641701 +0000 UTC m=+285.288060078 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls") pod "dns-default-vvwv5" (UID: "8efb9a7a-78b0-4351-8516-1af11cbe34a5") : secret "dns-default-metrics-tls" not found
Jul 25 00:38:28.576837 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:38:28.576676    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls podName:1dc22a0b-458b-4389-b3ef-f6ccf48c5de5 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:30.576662123 +0000 UTC m=+285.288080500 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls") pod "image-registry-84c5f665fb-xk66n" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5") : secret "image-registry-tls" not found
Jul 25 00:38:37.926677 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:38:37.926637    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:39:57.677563 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:39:57.677525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:39:57.678023 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:39:57.677604    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 25 00:39:57.678023 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:39:57.677671    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs podName:c67c6b28-9884-4498-bd1f-0b5f7d52231f nodeName:}" failed. No retries permitted until 2026-07-25 00:41:59.677656993 +0000 UTC m=+374.389075369 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs") pod "network-metrics-daemon-wdr58" (UID: "c67c6b28-9884-4498-bd1f-0b5f7d52231f") : secret "metrics-daemon-secret" not found
Jul 25 00:40:08.424998 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.424918    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"]
Jul 25 00:40:08.427595 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.427575    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"]
Jul 25 00:40:08.427689 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.427661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.430298 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.430275    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zb5ns"]
Jul 25 00:40:08.430388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.430347    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.433065 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.433044    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n8sw9"]
Jul 25 00:40:08.433154 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.433141    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.434127 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.434105    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 25 00:40:08.434246 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.434171    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 25 00:40:08.434338 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.434319    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jhhdd\""
Jul 25 00:40:08.435687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.435667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.436055 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436039    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.436140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436081    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 25 00:40:08.436140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436094    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rx2k5\""
Jul 25 00:40:08.436140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436045    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.436269 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436212    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.436704 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436688    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.436779 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436721    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 25 00:40:08.436779 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436725    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.436779 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436751    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 25 00:40:08.436931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436862    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 25 00:40:08.436931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436881    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4bgpt\""
Jul 25 00:40:08.436931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.436880    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.440820 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.440226    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 25 00:40:08.446072 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.446052    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"]
Jul 25 00:40:08.447014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.446566    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.447014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.446794    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mzdkm\""
Jul 25 00:40:08.447014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.446800    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 25 00:40:08.447820 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.447552    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.450207 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.450190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zb5ns"]
Jul 25 00:40:08.451245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.451209    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 25 00:40:08.451391 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.451366    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n8sw9"]
Jul 25 00:40:08.452240 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.452219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0b9f0f3-700d-4804-9885-87868abc05df-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.452442 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.452246    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 25 00:40:08.452560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.452424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62lqj\" (UniqueName: \"kubernetes.io/projected/b0b9f0f3-700d-4804-9885-87868abc05df-kube-api-access-62lqj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.452738 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.452712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0b9f0f3-700d-4804-9885-87868abc05df-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.453534 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.453503    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"]
Jul 25 00:40:08.518605 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.518579    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"]
Jul 25 00:40:08.521436 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.521422    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77c7dfb8cc-56vkm"]
Jul 25 00:40:08.521558 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.521542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.523787 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.523766    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 25 00:40:08.524016 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.524004    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.524073 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.524037    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.524229 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.524213    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.524711 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.524555    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 25 00:40:08.524839 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.524755    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-whrrt\""
Jul 25 00:40:08.526328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526312    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d2j86\""
Jul 25 00:40:08.526419 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526313    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 25 00:40:08.526475 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526459    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 25 00:40:08.526574 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526546    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 25 00:40:08.526773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526757    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 25 00:40:08.526932 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526896    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 25 00:40:08.527004 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.526963    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 25 00:40:08.531656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.531638    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"]
Jul 25 00:40:08.535607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.535592    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77c7dfb8cc-56vkm"]
Jul 25 00:40:08.553378 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553358    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-tmp\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.553485 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553406    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7l6x\" (UniqueName: \"kubernetes.io/projected/1ec34bf5-1799-4d44-9e40-841748a0401f-kube-api-access-x7l6x\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.553485 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553434    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7r7\" (UniqueName: \"kubernetes.io/projected/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-kube-api-access-xs7r7\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.553485 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-stats-auth\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.553663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553501    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62lqj\" (UniqueName: \"kubernetes.io/projected/b0b9f0f3-700d-4804-9885-87868abc05df-kube-api-access-62lqj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.553663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0b9f0f3-700d-4804-9885-87868abc05df-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.553663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-config\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.553663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.553663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d17a42-c88c-4260-87e6-bd0d9a8618e5-serving-cert\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.553833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-service-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.553833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zft4x\" (UniqueName: \"kubernetes.io/projected/00d17a42-c88c-4260-87e6-bd0d9a8618e5-kube-api-access-zft4x\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.553833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-default-certificate\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.553833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-config\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.553833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35fece75-0dd8-41f9-a7bd-0656c707d5b5-serving-cert\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-snapshots\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z76fm\" (UniqueName: \"kubernetes.io/projected/35fece75-0dd8-41f9-a7bd-0656c707d5b5-kube-api-access-z76fm\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-trusted-ca\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.554007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.553984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1947fb06-ef29-4fa6-8630-32bafb55d402-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.554240 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.554016    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xscm\" (UniqueName: \"kubernetes.io/projected/1947fb06-ef29-4fa6-8630-32bafb55d402-kube-api-access-4xscm\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.554240 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.554043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-serving-cert\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.554240 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.554067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0b9f0f3-700d-4804-9885-87868abc05df-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.554502 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.554483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0b9f0f3-700d-4804-9885-87868abc05df-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.555753 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.555734    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0b9f0f3-700d-4804-9885-87868abc05df-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.561744 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.561726    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62lqj\" (UniqueName: \"kubernetes.io/projected/b0b9f0f3-700d-4804-9885-87868abc05df-kube-api-access-62lqj\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6\" (UID: \"b0b9f0f3-700d-4804-9885-87868abc05df\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.654581 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.654581 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-trusted-ca\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1947fb06-ef29-4fa6-8630-32bafb55d402-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654618    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xscm\" (UniqueName: \"kubernetes.io/projected/1947fb06-ef29-4fa6-8630-32bafb55d402-kube-api-access-4xscm\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-serving-cert\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-tmp\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7l6x\" (UniqueName: \"kubernetes.io/projected/1ec34bf5-1799-4d44-9e40-841748a0401f-kube-api-access-x7l6x\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:08.654715    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 25 00:40:08.654811 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:08.654784    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:09.15476467 +0000 UTC m=+263.866183051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : secret "router-metrics-certs-default" not found
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654813    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7r7\" (UniqueName: \"kubernetes.io/projected/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-kube-api-access-xs7r7\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-stats-auth\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-config\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.654977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d17a42-c88c-4260-87e6-bd0d9a8618e5-serving-cert\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-service-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zft4x\" (UniqueName: \"kubernetes.io/projected/00d17a42-c88c-4260-87e6-bd0d9a8618e5-kube-api-access-zft4x\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-default-certificate\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-config\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655129    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35fece75-0dd8-41f9-a7bd-0656c707d5b5-serving-cert\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.655166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-snapshots\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655207    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-tmp\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655242    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z76fm\" (UniqueName: \"kubernetes.io/projected/35fece75-0dd8-41f9-a7bd-0656c707d5b5-kube-api-access-z76fm\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655616    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-config\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.655723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655618    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-service-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.656007 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.655837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35fece75-0dd8-41f9-a7bd-0656c707d5b5-trusted-ca\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.656093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d17a42-c88c-4260-87e6-bd0d9a8618e5-snapshots\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:08.656140    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:09.156123274 +0000 UTC m=+263.867541663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:08.656182    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:08.656229    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:09.156218147 +0000 UTC m=+263.867636528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.656376    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d17a42-c88c-4260-87e6-bd0d9a8618e5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.656756    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-config\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.656802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.656775    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1947fb06-ef29-4fa6-8630-32bafb55d402-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.657363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.657343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-serving-cert\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.657726 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.657702    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-stats-auth\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.657887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.657863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35fece75-0dd8-41f9-a7bd-0656c707d5b5-serving-cert\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.658081 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.658064    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-default-certificate\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.658416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.658401    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d17a42-c88c-4260-87e6-bd0d9a8618e5-serving-cert\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.663635 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.663613    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7r7\" (UniqueName: \"kubernetes.io/projected/3fe854e2-c0e9-4e36-8d35-ab77a8aee495-kube-api-access-xs7r7\") pod \"service-ca-operator-5d6c75f84f-dgktj\" (UID: \"3fe854e2-c0e9-4e36-8d35-ab77a8aee495\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.664706 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.664605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7l6x\" (UniqueName: \"kubernetes.io/projected/1ec34bf5-1799-4d44-9e40-841748a0401f-kube-api-access-x7l6x\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:08.664938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.664918    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xscm\" (UniqueName: \"kubernetes.io/projected/1947fb06-ef29-4fa6-8630-32bafb55d402-kube-api-access-4xscm\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:08.665493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.665473    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zft4x\" (UniqueName: \"kubernetes.io/projected/00d17a42-c88c-4260-87e6-bd0d9a8618e5-kube-api-access-zft4x\") pod \"insights-operator-5885fdb759-n8sw9\" (UID: \"00d17a42-c88c-4260-87e6-bd0d9a8618e5\") " pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.665493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.665484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z76fm\" (UniqueName: \"kubernetes.io/projected/35fece75-0dd8-41f9-a7bd-0656c707d5b5-kube-api-access-z76fm\") pod \"console-operator-584dcb6b54-zb5ns\" (UID: \"35fece75-0dd8-41f9-a7bd-0656c707d5b5\") " pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.667786 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.667752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:08.736665 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.736643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"
Jul 25 00:40:08.755386 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.755356    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:08.756916 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.756894    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:08.761401 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.761375    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-n8sw9"
Jul 25 00:40:08.831560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.831424    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"
Jul 25 00:40:08.879922 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.879848    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6"]
Jul 25 00:40:08.884191 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:08.884139    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b9f0f3_700d_4804_9885_87868abc05df.slice/crio-b6372f77ab178389685502117e3b69fa750652467ef5884545354636a7d1ce06 WatchSource:0}: Error finding container b6372f77ab178389685502117e3b69fa750652467ef5884545354636a7d1ce06: Status 404 returned error can't find the container with id b6372f77ab178389685502117e3b69fa750652467ef5884545354636a7d1ce06
Jul 25 00:40:08.908615 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.908256    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zb5ns"]
Jul 25 00:40:08.914174 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:08.914150    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fece75_0dd8_41f9_a7bd_0656c707d5b5.slice/crio-dc8921390031e798cc381bc1a0cc6b90eae903cde5011c8c8a2314335df879b1 WatchSource:0}: Error finding container dc8921390031e798cc381bc1a0cc6b90eae903cde5011c8c8a2314335df879b1: Status 404 returned error can't find the container with id dc8921390031e798cc381bc1a0cc6b90eae903cde5011c8c8a2314335df879b1
Jul 25 00:40:08.926008 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.925984    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-n8sw9"]
Jul 25 00:40:08.929010 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:08.928985    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d17a42_c88c_4260_87e6_bd0d9a8618e5.slice/crio-1032aee85d24ff0f8930b1085dbfedb371307a3c5d19abc970f018ddad2207d6 WatchSource:0}: Error finding container 1032aee85d24ff0f8930b1085dbfedb371307a3c5d19abc970f018ddad2207d6: Status 404 returned error can't find the container with id 1032aee85d24ff0f8930b1085dbfedb371307a3c5d19abc970f018ddad2207d6
Jul 25 00:40:08.964965 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:08.964942    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj"]
Jul 25 00:40:08.969434 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:08.969410    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe854e2_c0e9_4e36_8d35_ab77a8aee495.slice/crio-87f55871716e4d66fc7633ab02e685cda936b034aa67be732f84636d99f59004 WatchSource:0}: Error finding container 87f55871716e4d66fc7633ab02e685cda936b034aa67be732f84636d99f59004: Status 404 returned error can't find the container with id 87f55871716e4d66fc7633ab02e685cda936b034aa67be732f84636d99f59004
Jul 25 00:40:09.158691 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.158629    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:09.158691 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.158674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:09.158809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.158715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:09.158809 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:09.158782    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:10.158765379 +0000 UTC m=+264.870183756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:09.158909 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:09.158852    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 25 00:40:09.158909 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:09.158901    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:10.158889277 +0000 UTC m=+264.870307654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : secret "router-metrics-certs-default" not found
Jul 25 00:40:09.158973 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:09.158852    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:09.158973 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:09.158932    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:10.158923674 +0000 UTC m=+264.870342052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:09.738912 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.738846    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n8sw9" event={"ID":"00d17a42-c88c-4260-87e6-bd0d9a8618e5","Type":"ContainerStarted","Data":"1032aee85d24ff0f8930b1085dbfedb371307a3c5d19abc970f018ddad2207d6"}
Jul 25 00:40:09.740837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.740780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6" event={"ID":"b0b9f0f3-700d-4804-9885-87868abc05df","Type":"ContainerStarted","Data":"b6372f77ab178389685502117e3b69fa750652467ef5884545354636a7d1ce06"}
Jul 25 00:40:09.742341 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.742311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj" event={"ID":"3fe854e2-c0e9-4e36-8d35-ab77a8aee495","Type":"ContainerStarted","Data":"87f55871716e4d66fc7633ab02e685cda936b034aa67be732f84636d99f59004"}
Jul 25 00:40:09.743885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:09.743829    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" event={"ID":"35fece75-0dd8-41f9-a7bd-0656c707d5b5","Type":"ContainerStarted","Data":"dc8921390031e798cc381bc1a0cc6b90eae903cde5011c8c8a2314335df879b1"}
Jul 25 00:40:10.170057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:10.169983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:10.170057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:10.170032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:10.170313 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:10.170080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:10.170313 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:10.170207    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 25 00:40:10.170313 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:10.170294    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:12.170257605 +0000 UTC m=+266.881675987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : secret "router-metrics-certs-default" not found
Jul 25 00:40:10.170726 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:10.170707    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:12.170691217 +0000 UTC m=+266.882109598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:10.170819 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:10.170783    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:10.170819 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:10.170818    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:12.17080787 +0000 UTC m=+266.882226253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:10.393157 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:10.393123    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:11.894717 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:11.894669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:12.187977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.187897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:12.187977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.187942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:12.188182 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.187980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:12.188182 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:12.188058    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:16.188040382 +0000 UTC m=+270.899458764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:12.188182 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:12.188128    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 25 00:40:12.188314 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:12.188186    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:16.188172401 +0000 UTC m=+270.899590777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : secret "router-metrics-certs-default" not found
Jul 25 00:40:12.188314 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:12.188129    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:12.188314 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:12.188221    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:16.188212393 +0000 UTC m=+270.899630775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:12.257993 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.257952    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:12.358951 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.358921    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:12.754701 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.754630    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n8sw9" event={"ID":"00d17a42-c88c-4260-87e6-bd0d9a8618e5","Type":"ContainerStarted","Data":"67671f51510641121571203a7554a70da8fc7a5b9854a361f2e3f09c4f9d3b8b"}
Jul 25 00:40:12.756229 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.756207    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6" event={"ID":"b0b9f0f3-700d-4804-9885-87868abc05df","Type":"ContainerStarted","Data":"8320ccfdcf5f4a722d8e6899fb6c3ee722fd56f9e72f22ba360eca0ae9426e66"}
Jul 25 00:40:12.757660 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.757634    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj" event={"ID":"3fe854e2-c0e9-4e36-8d35-ab77a8aee495","Type":"ContainerStarted","Data":"f1c3afb858d93dcf4d630bd49789199fe7e51eb25174eea9637aab1831cb8f6b"}
Jul 25 00:40:12.759304 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.759286    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/0.log"
Jul 25 00:40:12.759407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.759317    2574 generic.go:356] "Generic (PLEG): container finished" podID="35fece75-0dd8-41f9-a7bd-0656c707d5b5" containerID="0c865eee6eb812feccaf2a0c8c529321d040e05119255ee79839e2ad9b73c3ab" exitCode=255
Jul 25 00:40:12.759407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.759343    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" event={"ID":"35fece75-0dd8-41f9-a7bd-0656c707d5b5","Type":"ContainerDied","Data":"0c865eee6eb812feccaf2a0c8c529321d040e05119255ee79839e2ad9b73c3ab"}
Jul 25 00:40:12.759608 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.759591    2574 scope.go:117] "RemoveContainer" containerID="0c865eee6eb812feccaf2a0c8c529321d040e05119255ee79839e2ad9b73c3ab"
Jul 25 00:40:12.770563 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.770505    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-n8sw9" podStartSLOduration=1.372515068 podStartE2EDuration="4.770489193s" totalImagesPullingTime="3.397974125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:12.769738521 +0000 UTC m=+267.481156921" watchObservedRunningTime="2026-07-25 00:40:12.770489193 +0000 UTC m=+267.481907595"
Jul 25 00:40:12.791360 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.791298    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6" podStartSLOduration=1.353140926 podStartE2EDuration="4.79128106s" totalImagesPullingTime="3.438140134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:12.783481053 +0000 UTC m=+267.494899452" watchObservedRunningTime="2026-07-25 00:40:12.79128106 +0000 UTC m=+267.502699460"
Jul 25 00:40:12.800420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:12.800367    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj" podStartSLOduration=1.446432579 podStartE2EDuration="4.800352255s" totalImagesPullingTime="3.353919676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:12.800222573 +0000 UTC m=+267.511640971" watchObservedRunningTime="2026-07-25 00:40:12.800352255 +0000 UTC m=+267.511770655"
Jul 25 00:40:13.219136 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.219096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:13.275628 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.275571    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:13.763662 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.763637    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:40:13.764059 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.764046    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/0.log"
Jul 25 00:40:13.764130 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.764080    2574 generic.go:356] "Generic (PLEG): container finished" podID="35fece75-0dd8-41f9-a7bd-0656c707d5b5" containerID="383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017" exitCode=255
Jul 25 00:40:13.764187 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.764156    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" event={"ID":"35fece75-0dd8-41f9-a7bd-0656c707d5b5","Type":"ContainerDied","Data":"383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017"}
Jul 25 00:40:13.764260 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.764245    2574 scope.go:117] "RemoveContainer" containerID="0c865eee6eb812feccaf2a0c8c529321d040e05119255ee79839e2ad9b73c3ab"
Jul 25 00:40:13.764548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:13.764530    2574 scope.go:117] "RemoveContainer" containerID="383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017"
Jul 25 00:40:13.764714 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:13.764696    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zb5ns_openshift-console-operator(35fece75-0dd8-41f9-a7bd-0656c707d5b5)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" podUID="35fece75-0dd8-41f9-a7bd-0656c707d5b5"
Jul 25 00:40:14.147203 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.147116    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:14.198530 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.198481    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:14.227709 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.227681    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bvxkg"]
Jul 25 00:40:14.232536 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.232492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.235444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.234824    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 25 00:40:14.235444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.235092    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 25 00:40:14.235444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.235291    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x25j\""
Jul 25 00:40:14.242847 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.242825    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bvxkg"]
Jul 25 00:40:14.264977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.264952    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:14.312162 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.312140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7e3e027-f042-4489-8d36-711e9fe930eb-crio-socket\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.312262 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.312220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7e3e027-f042-4489-8d36-711e9fe930eb-data-volume\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.312324 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.312308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.312374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.312360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8wgm\" (UniqueName: \"kubernetes.io/projected/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-api-access-x8wgm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.312463 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.312432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.377585 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.377564    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:14.413541 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.413641 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8wgm\" (UniqueName: \"kubernetes.io/projected/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-api-access-x8wgm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.413641 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.413578    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.413641 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.413641 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.413641    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:14.913625343 +0000 UTC m=+269.625043737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.413846 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7e3e027-f042-4489-8d36-711e9fe930eb-crio-socket\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.413846 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.413698    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.413846 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7e3e027-f042-4489-8d36-711e9fe930eb-data-volume\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.413846 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.413747    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:14.913737048 +0000 UTC m=+269.625155426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.413846 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.413769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7e3e027-f042-4489-8d36-711e9fe930eb-crio-socket\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.414078 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.414014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7e3e027-f042-4489-8d36-711e9fe930eb-data-volume\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.423174 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.423147    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8wgm\" (UniqueName: \"kubernetes.io/projected/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-api-access-x8wgm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.478416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.478389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:14.768992 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.768967    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:40:14.769256 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.769240    2574 scope.go:117] "RemoveContainer" containerID="383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017"
Jul 25 00:40:14.769386 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.769367    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zb5ns_openshift-console-operator(35fece75-0dd8-41f9-a7bd-0656c707d5b5)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" podUID="35fece75-0dd8-41f9-a7bd-0656c707d5b5"
Jul 25 00:40:14.917213 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.917183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.917357 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.917255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:14.917357 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.917339    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.917469 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.917394    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.917469 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.917406    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:15.91738606 +0000 UTC m=+270.628804443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:14.917469 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:14.917456    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:15.917432912 +0000 UTC m=+270.628851288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:14.927610 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:14.927584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:15.027551 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:15.027451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:15.926910 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:15.926879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:15.927277 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:15.926985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:15.927277 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:15.926991    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:15.927277 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:15.927067    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:17.927046367 +0000 UTC m=+272.638464746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:15.927277 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:15.927068    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:15.927277 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:15.927112    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:17.927097581 +0000 UTC m=+272.638515958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:16.229152 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.229124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.229155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.229186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:16.229276    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:16.229285    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:24.229267058 +0000 UTC m=+278.940685446 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:16.229276    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:16.229315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:16.229317    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:24.229306492 +0000 UTC m=+278.940724869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : secret "router-metrics-certs-default" not found
Jul 25 00:40:16.229603 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:16.229338    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:24.229328689 +0000 UTC m=+278.940747081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:16.420001 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.419972    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:16.451351 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.451325    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-4bssh"]
Jul 25 00:40:16.454314 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.454299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.456538 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.456501    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 25 00:40:16.457430 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.457410    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 25 00:40:16.457531 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.457493    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 25 00:40:16.457531 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.457522    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jlb64\""
Jul 25 00:40:16.457653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.457497    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 25 00:40:16.461178 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.461159    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-4bssh"]
Jul 25 00:40:16.478111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.478091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:16.531107 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.531057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-cabundle\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.531186 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.531123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpz4v\" (UniqueName: \"kubernetes.io/projected/08a95064-1e2c-4963-a45a-0914492f9fb0-kube-api-access-mpz4v\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.531186 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.531178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-key\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.550329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.550303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:16.631689 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.631668    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpz4v\" (UniqueName: \"kubernetes.io/projected/08a95064-1e2c-4963-a45a-0914492f9fb0-kube-api-access-mpz4v\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.631784 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.631721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-key\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.631899 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.631880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-cabundle\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.632428 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.632404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-cabundle\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.633924 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.633899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/08a95064-1e2c-4963-a45a-0914492f9fb0-signing-key\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.638528 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.638487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpz4v\" (UniqueName: \"kubernetes.io/projected/08a95064-1e2c-4963-a45a-0914492f9fb0-kube-api-access-mpz4v\") pod \"service-ca-658d7b7c78-4bssh\" (UID: \"08a95064-1e2c-4963-a45a-0914492f9fb0\") " pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.650796 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.650775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:16.763620 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.763602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-4bssh"
Jul 25 00:40:16.880351 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:16.880324    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-4bssh"]
Jul 25 00:40:16.882913 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:16.882888    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a95064_1e2c_4963_a45a_0914492f9fb0.slice/crio-780015cb5997d416864163a73e60b39102900116e297c6bee78692da0d0482fc WatchSource:0}: Error finding container 780015cb5997d416864163a73e60b39102900116e297c6bee78692da0d0482fc: Status 404 returned error can't find the container with id 780015cb5997d416864163a73e60b39102900116e297c6bee78692da0d0482fc
Jul 25 00:40:17.021471 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.021449    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k5qjn_41a7e68e-c7eb-4150-9b35-424eaf1cb3f7/dns-node-resolver/0.log"
Jul 25 00:40:17.328595 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.328565    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:17.779506 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.779475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-4bssh" event={"ID":"08a95064-1e2c-4963-a45a-0914492f9fb0","Type":"ContainerStarted","Data":"8c49ee343d3a72bbfb63e1996f316410994bbf845bd99670ccf0a62b30b0d256"}
Jul 25 00:40:17.779674 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.779538    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-4bssh" event={"ID":"08a95064-1e2c-4963-a45a-0914492f9fb0","Type":"ContainerStarted","Data":"780015cb5997d416864163a73e60b39102900116e297c6bee78692da0d0482fc"}
Jul 25 00:40:17.795021 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.794978    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-4bssh" podStartSLOduration=1.794965596 podStartE2EDuration="1.794965596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:17.794492893 +0000 UTC m=+272.505911294" watchObservedRunningTime="2026-07-25 00:40:17.794965596 +0000 UTC m=+272.506383995"
Jul 25 00:40:17.942477 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.942452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:17.942618 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:17.942504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:17.942618 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:17.942576    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:17.942688 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:17.942632    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:17.942688 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:17.942647    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:21.942626942 +0000 UTC m=+276.654045322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:17.942688 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:17.942667    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:21.942656818 +0000 UTC m=+276.654075195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:18.167577 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.167485    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:18.223969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.223944    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5qb9_59310f30-69f1-4d8b-8939-2320ae3ca7b4/node-ca/0.log"
Jul 25 00:40:18.285607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.285571    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:18.343653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.343628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:18.597702 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.597670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:18.699455 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.699370    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:18.755604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.755582    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:18.755753 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.755630    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:18.755943 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.755931    2574 scope.go:117] "RemoveContainer" containerID="383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017"
Jul 25 00:40:18.756092 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:18.756075    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zb5ns_openshift-console-operator(35fece75-0dd8-41f9-a7bd-0656c707d5b5)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" podUID="35fece75-0dd8-41f9-a7bd-0656c707d5b5"
Jul 25 00:40:18.866100 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:18.866073    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:19.022120 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:19.022098    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzzs5_128b0291-5428-46d8-b2dd-fa8aeb05fe30/serve-healthcheck-canary/0.log"
Jul 25 00:40:19.623892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:19.623860    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6_b0b9f0f3-700d-4804-9885-87868abc05df/kube-storage-version-migrator-operator/0.log"
Jul 25 00:40:19.679678 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:19.679652    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:19.772139 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:19.772113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:20.279970 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:20.279941    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:20.474732 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:20.474697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:20.663568 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:20.663479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:21.066727 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:21.066689    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:21.852275 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:21.852240    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:21.953763 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:21.953716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:21.974556 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:21.974528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:21.974656 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:21.974580    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 00:40:21.974656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:21.974609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:21.974656 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:21.974634    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:29.97461725 +0000 UTC m=+284.686035635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : configmap "kube-rbac-proxy" not found
Jul 25 00:40:21.974770 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:21.974696    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:21.974770 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:21.974742    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls podName:f7e3e027-f042-4489-8d36-711e9fe930eb nodeName:}" failed. No retries permitted until 2026-07-25 00:40:29.974731833 +0000 UTC m=+284.686150211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvxkg" (UID: "f7e3e027-f042-4489-8d36-711e9fe930eb") : secret "insights-runtime-extractor-tls" not found
Jul 25 00:40:22.070420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:22.070392    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:22.469786 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:22.469749    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:22.672402 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:22.672339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:23.066574 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:23.066532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:23.467198 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:23.467166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:24.015037 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.015004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:24.112452 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.112414    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:24.292565 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.292484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:24.292565 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.292531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:24.292565 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.292564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:24.292768 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:24.292646    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle podName:1ec34bf5-1799-4d44-9e40-841748a0401f nodeName:}" failed. No retries permitted until 2026-07-25 00:40:40.29262951 +0000 UTC m=+295.004047887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle") pod "router-default-77c7dfb8cc-56vkm" (UID: "1ec34bf5-1799-4d44-9e40-841748a0401f") : configmap references non-existent config key: service-ca.crt
Jul 25 00:40:24.292768 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:24.292695    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:24.292768 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:24.292752    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls podName:1947fb06-ef29-4fa6-8630-32bafb55d402 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:40.292739975 +0000 UTC m=+295.004158352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jkkmr" (UID: "1947fb06-ef29-4fa6-8630-32bafb55d402") : secret "cluster-monitoring-operator-tls" not found
Jul 25 00:40:24.294897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.294869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ec34bf5-1799-4d44-9e40-841748a0401f-metrics-certs\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:24.676729 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.676641    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:24.732424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.732391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:24.867870 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:24.867843    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:25.267611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:25.267578    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:25.468326 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:25.468287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:25.667922 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:25.667848    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:26.312348 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:26.312315    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:26.866555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:26.866504    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:27.070976 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.070943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:27.266255 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.266216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:27.462475 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.462441    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:27.499555 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:27.499500    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vvwv5" podUID="8efb9a7a-78b0-4351-8516-1af11cbe34a5"
Jul 25 00:40:27.499555 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:27.499532    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9" podUID="0fb6c2c8-80c7-467a-86b5-062613ab1f15"
Jul 25 00:40:27.499555 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:27.499529    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"
Jul 25 00:40:27.668715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.668604    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:27.806322 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.806292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:27.806484 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.806298    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:40:27.806484 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:27.806299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:28.463856 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:28.463813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:28.662737 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:28.662701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:28.868379 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:28.868294    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:30.039745 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.039711    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:30.040139 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.039799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:30.040270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.040253    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7e3e027-f042-4489-8d36-711e9fe930eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:30.041972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.041955    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7e3e027-f042-4489-8d36-711e9fe930eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvxkg\" (UID: \"f7e3e027-f042-4489-8d36-711e9fe930eb\") " pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:30.143008 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.142952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bvxkg"
Jul 25 00:40:30.266824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.266791    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bvxkg"]
Jul 25 00:40:30.269568 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:30.269544    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e3e027_f042_4489_8d36_711e9fe930eb.slice/crio-230ec570c9f8d1e0d70c572d371177b039b98aa21b8ee3e9845b000007df2b20 WatchSource:0}: Error finding container 230ec570c9f8d1e0d70c572d371177b039b98aa21b8ee3e9845b000007df2b20: Status 404 returned error can't find the container with id 230ec570c9f8d1e0d70c572d371177b039b98aa21b8ee3e9845b000007df2b20
Jul 25 00:40:30.645137 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.645093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:30.645344 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.645238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:30.645344 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.645289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:40:30.647978 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.647951    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fb6c2c8-80c7-467a-86b5-062613ab1f15-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-xv9g9\" (UID: \"0fb6c2c8-80c7-467a-86b5-062613ab1f15\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:40:30.648118 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.647953    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8efb9a7a-78b0-4351-8516-1af11cbe34a5-metrics-tls\") pod \"dns-default-vvwv5\" (UID: \"8efb9a7a-78b0-4351-8516-1af11cbe34a5\") " pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:30.648118 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.648019    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"image-registry-84c5f665fb-xk66n\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") " pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:30.810739 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.810322    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d68th\""
Jul 25 00:40:30.810739 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.810575    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b2m9l\""
Jul 25 00:40:30.810739 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.810663    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5x988\""
Jul 25 00:40:30.817239 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.817125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"
Jul 25 00:40:30.817239 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.817161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:30.817437 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.817245    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:30.818199 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:30.818171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvxkg" event={"ID":"f7e3e027-f042-4489-8d36-711e9fe930eb","Type":"ContainerStarted","Data":"230ec570c9f8d1e0d70c572d371177b039b98aa21b8ee3e9845b000007df2b20"}
Jul 25 00:40:31.004123 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.003993    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9"]
Jul 25 00:40:31.027733 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.026531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvwv5"]
Jul 25 00:40:31.033045 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:31.033017    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efb9a7a_78b0_4351_8516_1af11cbe34a5.slice/crio-c082ce8b2eef4cb6e2aece73537962249e9f665323a0dbd7d2746a818781b140 WatchSource:0}: Error finding container c082ce8b2eef4cb6e2aece73537962249e9f665323a0dbd7d2746a818781b140: Status 404 returned error can't find the container with id c082ce8b2eef4cb6e2aece73537962249e9f665323a0dbd7d2746a818781b140
Jul 25 00:40:31.057394 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.057326    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:40:31.062985 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:31.062958    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc22a0b_458b_4389_b3ef_f6ccf48c5de5.slice/crio-36ef243b462067509e6c68f16982b07135aeb93c3662e6a04b35a4aefb4371f2 WatchSource:0}: Error finding container 36ef243b462067509e6c68f16982b07135aeb93c3662e6a04b35a4aefb4371f2: Status 404 returned error can't find the container with id 36ef243b462067509e6c68f16982b07135aeb93c3662e6a04b35a4aefb4371f2
Jul 25 00:40:31.824894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.824853    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" event={"ID":"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5","Type":"ContainerStarted","Data":"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"}
Jul 25 00:40:31.825093 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.824924    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:31.825093 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.824944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" event={"ID":"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5","Type":"ContainerStarted","Data":"36ef243b462067509e6c68f16982b07135aeb93c3662e6a04b35a4aefb4371f2"}
Jul 25 00:40:31.826495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.826452    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9" event={"ID":"0fb6c2c8-80c7-467a-86b5-062613ab1f15","Type":"ContainerStarted","Data":"5480b027589b641584da6644e3c26ec8583dac77ee884ec1d754b734e212c5b0"}
Jul 25 00:40:31.828714 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.828654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvwv5" event={"ID":"8efb9a7a-78b0-4351-8516-1af11cbe34a5","Type":"ContainerStarted","Data":"c082ce8b2eef4cb6e2aece73537962249e9f665323a0dbd7d2746a818781b140"}
Jul 25 00:40:31.843048 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:31.842818    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" podStartSLOduration=284.842800456 podStartE2EDuration="4m44.842800456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:31.842107263 +0000 UTC m=+286.553525676" watchObservedRunningTime="2026-07-25 00:40:31.842800456 +0000 UTC m=+286.554218853"
Jul 25 00:40:32.834199 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:32.834161    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9" event={"ID":"0fb6c2c8-80c7-467a-86b5-062613ab1f15","Type":"ContainerStarted","Data":"2057821e0527a7715a9207c9b8f1f3c4f2abfd6506ec6b208d75b015e935a2e1"}
Jul 25 00:40:32.849556 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:32.849483    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-xv9g9" podStartSLOduration=265.526397304 podStartE2EDuration="4m26.849465079s" totalImagesPullingTime="1.323067775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:32.848388294 +0000 UTC m=+287.559806716" watchObservedRunningTime="2026-07-25 00:40:32.849465079 +0000 UTC m=+287.560883478"
Jul 25 00:40:33.927549 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:33.927505    2574 scope.go:117] "RemoveContainer" containerID="383f93d6a8a342b4283d4ace17ec19e46a50ab17f5ba47ac258d15e893ea9017"
Jul 25 00:40:34.429631 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:34.429592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:34.531731 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:34.531692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:34.627938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:34.627877    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:34.774960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:34.774930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:34.828842 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:34.828807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:35.158884 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.158791    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:35.258192 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.258147    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:35.845348 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.845330    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:40:35.845448 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.845389    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" event={"ID":"35fece75-0dd8-41f9-a7bd-0656c707d5b5","Type":"ContainerStarted","Data":"c61e14514d94b12864392747eadc52937557a9f8bb301456d74e686e66e60bab"}
Jul 25 00:40:35.845741 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.845717    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:35.847018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.846996    2574 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-zb5ns container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused" start-of-body=
Jul 25 00:40:35.847107 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.847044    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" podUID="35fece75-0dd8-41f9-a7bd-0656c707d5b5" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused"
Jul 25 00:40:35.861922 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.861881    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns" podStartSLOduration=24.45035815 podStartE2EDuration="27.861863019s" totalImagesPullingTime="3.411504869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:35.861423821 +0000 UTC m=+290.572842220" watchObservedRunningTime="2026-07-25 00:40:35.861863019 +0000 UTC m=+290.573281420"
Jul 25 00:40:35.878724 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:35.878702    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:36.347726 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.347696    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-rhg4x"]
Jul 25 00:40:36.362727 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.362694    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:36.363431 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.363405    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-rhg4x"]
Jul 25 00:40:36.363554 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.363492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:36.365760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.365728    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 25 00:40:36.365873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.365792    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 25 00:40:36.365873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.365831    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-572cb\""
Jul 25 00:40:36.455629 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.455577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:36.500182 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.500154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fcvv\" (UniqueName: \"kubernetes.io/projected/64b7e67f-2fb5-43d9-af0a-23df633b13a3-kube-api-access-9fcvv\") pod \"downloads-7c97dc4555-rhg4x\" (UID: \"64b7e67f-2fb5-43d9-af0a-23df633b13a3\") " pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:36.600777 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.600708    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fcvv\" (UniqueName: \"kubernetes.io/projected/64b7e67f-2fb5-43d9-af0a-23df633b13a3-kube-api-access-9fcvv\") pod \"downloads-7c97dc4555-rhg4x\" (UID: \"64b7e67f-2fb5-43d9-af0a-23df633b13a3\") " pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:36.607875 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.607845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fcvv\" (UniqueName: \"kubernetes.io/projected/64b7e67f-2fb5-43d9-af0a-23df633b13a3-kube-api-access-9fcvv\") pod \"downloads-7c97dc4555-rhg4x\" (UID: \"64b7e67f-2fb5-43d9-af0a-23df633b13a3\") " pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:36.647880 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.647845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:36.671937 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.671914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:36.834993 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.834886    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-rhg4x"]
Jul 25 00:40:36.839058 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:36.839028    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b7e67f_2fb5_43d9_af0a_23df633b13a3.slice/crio-fb4a2102490266fe7880022908d5efe87c7ddc3aa72f7402f5da5501a8a3156e WatchSource:0}: Error finding container fb4a2102490266fe7880022908d5efe87c7ddc3aa72f7402f5da5501a8a3156e: Status 404 returned error can't find the container with id fb4a2102490266fe7880022908d5efe87c7ddc3aa72f7402f5da5501a8a3156e
Jul 25 00:40:36.850768 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.850734    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvwv5" event={"ID":"8efb9a7a-78b0-4351-8516-1af11cbe34a5","Type":"ContainerStarted","Data":"747d9d4f2e4fd7f479691ed6827695b764782f9a3a716439ac90ea71605358a1"}
Jul 25 00:40:36.850903 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.850796    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvwv5" event={"ID":"8efb9a7a-78b0-4351-8516-1af11cbe34a5","Type":"ContainerStarted","Data":"06fc77cdfb8ba8954456ba67162d5729e10350cd989568ffc0996d0d033a37c2"}
Jul 25 00:40:36.850903 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.850827    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:36.852005 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.851979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-rhg4x" event={"ID":"64b7e67f-2fb5-43d9-af0a-23df633b13a3","Type":"ContainerStarted","Data":"fb4a2102490266fe7880022908d5efe87c7ddc3aa72f7402f5da5501a8a3156e"}
Jul 25 00:40:36.853611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.853386    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvxkg" event={"ID":"f7e3e027-f042-4489-8d36-711e9fe930eb","Type":"ContainerStarted","Data":"030ad791eaac79bd3ebb0c416685c694ba450998ab97df86060607fc9f8bd391"}
Jul 25 00:40:36.857921 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.857897    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-zb5ns"
Jul 25 00:40:36.866139 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:36.866085    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vvwv5" podStartSLOduration=252.156465672 podStartE2EDuration="4m16.866066866s" totalImagesPullingTime="4.709601194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:36.864930543 +0000 UTC m=+291.576348941" watchObservedRunningTime="2026-07-25 00:40:36.866066866 +0000 UTC m=+291.577485266"
Jul 25 00:40:37.026827 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.026786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:37.077569 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.077535    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:37.457072 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.457042    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:37.503939 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.503875    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:37.654142 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.654105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:37.703352 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:37.703313    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:38.237098 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:38.237041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:38.440249 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:38.440188    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:38.806155 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:38.806092    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:39.667386 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:39.667338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:39.721107 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:39.721070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:40.333244 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.333205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:40.333686 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.333259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:40.333935 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.333914    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec34bf5-1799-4d44-9e40-841748a0401f-service-ca-bundle\") pod \"router-default-77c7dfb8cc-56vkm\" (UID: \"1ec34bf5-1799-4d44-9e40-841748a0401f\") " pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:40.336120 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.336092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1947fb06-ef29-4fa6-8630-32bafb55d402-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jkkmr\" (UID: \"1947fb06-ef29-4fa6-8630-32bafb55d402\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:40.336261 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.336246    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:40.542180 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.542149    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77c7dfb8cc-56vkm"]
Jul 25 00:40:40.545931 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:40.545905    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec34bf5_1799_4d44_9e40_841748a0401f.slice/crio-9756f3add3ac1d0fbae3050a23a69b75a6abedb4ea53a7fa978770be13bdf0d1 WatchSource:0}: Error finding container 9756f3add3ac1d0fbae3050a23a69b75a6abedb4ea53a7fa978770be13bdf0d1: Status 404 returned error can't find the container with id 9756f3add3ac1d0fbae3050a23a69b75a6abedb4ea53a7fa978770be13bdf0d1
Jul 25 00:40:40.546046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.546010    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"
Jul 25 00:40:40.696692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.696545    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr"]
Jul 25 00:40:40.699342 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:40.699315    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1947fb06_ef29_4fa6_8630_32bafb55d402.slice/crio-bfe81b77ac2ebb89703f28aaf8c08e850419a2a97b6de4a9e46e721f4f840b3b WatchSource:0}: Error finding container bfe81b77ac2ebb89703f28aaf8c08e850419a2a97b6de4a9e46e721f4f840b3b: Status 404 returned error can't find the container with id bfe81b77ac2ebb89703f28aaf8c08e850419a2a97b6de4a9e46e721f4f840b3b
Jul 25 00:40:40.872267 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.872181    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm" event={"ID":"1ec34bf5-1799-4d44-9e40-841748a0401f","Type":"ContainerStarted","Data":"dc9ea1a82d2d45e41a5c7bf8e6196220587b8da3eaa407e8faacdafba1a6e8f1"}
Jul 25 00:40:40.872267 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.872240    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm" event={"ID":"1ec34bf5-1799-4d44-9e40-841748a0401f","Type":"ContainerStarted","Data":"9756f3add3ac1d0fbae3050a23a69b75a6abedb4ea53a7fa978770be13bdf0d1"}
Jul 25 00:40:40.873626 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.873598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr" event={"ID":"1947fb06-ef29-4fa6-8630-32bafb55d402","Type":"ContainerStarted","Data":"bfe81b77ac2ebb89703f28aaf8c08e850419a2a97b6de4a9e46e721f4f840b3b"}
Jul 25 00:40:40.875790 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.875764    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvxkg" event={"ID":"f7e3e027-f042-4489-8d36-711e9fe930eb","Type":"ContainerStarted","Data":"a7f0aadec06a3f2eb05309216bbb7c631af168655779838b9a7befebb7240242"}
Jul 25 00:40:40.890375 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:40.890322    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm" podStartSLOduration=32.890307776 podStartE2EDuration="32.890307776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:40.888354197 +0000 UTC m=+295.599772607" watchObservedRunningTime="2026-07-25 00:40:40.890307776 +0000 UTC m=+295.601726175"
Jul 25 00:40:40.928002 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:40.927959    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-wdr58" podUID="c67c6b28-9884-4498-bd1f-0b5f7d52231f"
Jul 25 00:40:41.336956 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:41.336916    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:41.340070 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:41.339821    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:41.881990 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:41.881873    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:41.883622 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:41.883319    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77c7dfb8cc-56vkm"
Jul 25 00:40:44.897426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:44.897397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr" event={"ID":"1947fb06-ef29-4fa6-8630-32bafb55d402","Type":"ContainerStarted","Data":"172681d455b341102b0ed2a3417db61ec9b8a91a3b2b78b2ce8129837f0f3c06"}
Jul 25 00:40:44.911885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:44.911837    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jkkmr" podStartSLOduration=32.809068256 podStartE2EDuration="36.911821536s" totalImagesPullingTime="4.10275328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:44.910464357 +0000 UTC m=+299.621882756" watchObservedRunningTime="2026-07-25 00:40:44.911821536 +0000 UTC m=+299.623239934"
Jul 25 00:40:45.148923 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.148807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:45.249465 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.249397    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:45.307999 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.307972    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"]
Jul 25 00:40:45.311279 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.311253    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:45.313175 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.313113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:45.313595 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.313562    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 25 00:40:45.313705 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.313564    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-q4fgg\""
Jul 25 00:40:45.320731 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.320534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"]
Jul 25 00:40:45.491306 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.491270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/067d5345-5a0b-4d32-9127-515d3ba29144-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t6jjq\" (UID: \"067d5345-5a0b-4d32-9127-515d3ba29144\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:45.524018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.523972    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:45.592205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.592165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/067d5345-5a0b-4d32-9127-515d3ba29144-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t6jjq\" (UID: \"067d5345-5a0b-4d32-9127-515d3ba29144\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:45.594980 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.594948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/067d5345-5a0b-4d32-9127-515d3ba29144-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-t6jjq\" (UID: \"067d5345-5a0b-4d32-9127-515d3ba29144\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:45.622954 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.622926    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:45.710851 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.710825    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:40:45.712699 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.712655    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:40:45.744282 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.744220    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 25 00:40:45.770865 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.770830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"]
Jul 25 00:40:45.774630 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:45.774598    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067d5345_5a0b_4d32_9127_515d3ba29144.slice/crio-6f7039e9c53f967b456f3c53150a3542f5e2ba6702fd2c01dfdcf6a59ecf1256 WatchSource:0}: Error finding container 6f7039e9c53f967b456f3c53150a3542f5e2ba6702fd2c01dfdcf6a59ecf1256: Status 404 returned error can't find the container with id 6f7039e9c53f967b456f3c53150a3542f5e2ba6702fd2c01dfdcf6a59ecf1256
Jul 25 00:40:45.904044 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.903861    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvxkg" event={"ID":"f7e3e027-f042-4489-8d36-711e9fe930eb","Type":"ContainerStarted","Data":"9637244a757ef1383eb20e963237640e7fd334e170f435a325d95c27ad7b666b"}
Jul 25 00:40:45.930620 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.905113    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq" event={"ID":"067d5345-5a0b-4d32-9127-515d3ba29144","Type":"ContainerStarted","Data":"6f7039e9c53f967b456f3c53150a3542f5e2ba6702fd2c01dfdcf6a59ecf1256"}
Jul 25 00:40:45.930620 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:45.924209    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bvxkg" podStartSLOduration=17.570379765 podStartE2EDuration="31.924188742s" totalImagesPullingTime="14.353808977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:40:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:45.923865383 +0000 UTC m=+300.635283790" watchObservedRunningTime="2026-07-25 00:40:45.924188742 +0000 UTC m=+300.635607150"
Jul 25 00:40:46.860160 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.860132    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vvwv5"
Jul 25 00:40:46.911400 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.911356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq" event={"ID":"067d5345-5a0b-4d32-9127-515d3ba29144","Type":"ContainerStarted","Data":"25de70fca411039c0154c3cf928f6c2d432507ce9f9afd0111f87fc4c3c2e9cc"}
Jul 25 00:40:46.911956 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.911905    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:46.912637 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.912611    2574 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.19:8443/healthz\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 25 00:40:46.912742 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.912656    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq" podUID="067d5345-5a0b-4d32-9127-515d3ba29144" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.19:8443/healthz\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 25 00:40:46.926966 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:46.926871    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq" podStartSLOduration=0.873593394 podStartE2EDuration="1.926851817s" totalImagesPullingTime="1.053258423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:46.925565326 +0000 UTC m=+301.636983726" watchObservedRunningTime="2026-07-25 00:40:46.926851817 +0000 UTC m=+301.638270216"
Jul 25 00:40:47.919243 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:47.919212    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-t6jjq"
Jul 25 00:40:48.332326 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:48.332286    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:48.397051 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:48.397017    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:48.529883 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:48.529849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:50.822751 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:50.822715    2574 patch_prober.go:28] interesting pod/image-registry-84c5f665fb-xk66n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 25 00:40:50.823177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:50.822770    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 00:40:51.032140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:51.032102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:51.926271 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:51.926241    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:40:52.839972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:52.839885    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:40:53.085876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.085834    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.131666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.131557    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.255968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.255934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.303185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.303146    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.476211 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.476121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.686142 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.686075    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.724303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.724270    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dnwsz"]
Jul 25 00:40:53.761822 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.761710    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.780260 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.780234    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"]
Jul 25 00:40:53.781664 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.781646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.784476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.784454    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 25 00:40:53.784854 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.784783    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 25 00:40:53.784954 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.784852    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nql9s\""
Jul 25 00:40:53.785017 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.784961    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 25 00:40:53.785525 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.785492    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 25 00:40:53.802352 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.802331    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"]
Jul 25 00:40:53.802459 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.802447    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.804810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.804644    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 25 00:40:53.804810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.804682    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 25 00:40:53.805179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.805159    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-84nk2\""
Jul 25 00:40:53.805422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.805404    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868048    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/41811469-4ed5-4543-ba64-a907100bc569-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-sys\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-tls\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-root\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-wtmp\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868289    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bv5n\" (UniqueName: \"kubernetes.io/projected/76165ad8-fb19-412c-96f0-bb972c9d72b6-kube-api-access-2bv5n\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-metrics-client-ca\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-textfile\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868674    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xvzh\" (UniqueName: \"kubernetes.io/projected/41811469-4ed5-4543-ba64-a907100bc569-kube-api-access-2xvzh\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.868816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.868700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.928598 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.928334    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:53.969423 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-textfile\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xvzh\" (UniqueName: \"kubernetes.io/projected/41811469-4ed5-4543-ba64-a907100bc569-kube-api-access-2xvzh\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/41811469-4ed5-4543-ba64-a907100bc569-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969557    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-sys\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-tls\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969607    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-root\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-wtmp\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969679    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bv5n\" (UniqueName: \"kubernetes.io/projected/76165ad8-fb19-412c-96f0-bb972c9d72b6-kube-api-access-2bv5n\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.969866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.969823    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-metrics-client-ca\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.970479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.970454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-metrics-client-ca\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.970737 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.970716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-textfile\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.971487 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.971461    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-wtmp\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.971596 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:53.971485    2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 25 00:40:53.971596 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:53.971568    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls podName:41811469-4ed5-4543-ba64-a907100bc569 nodeName:}" failed. No retries permitted until 2026-07-25 00:40:54.471548387 +0000 UTC m=+309.182966780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-v95ws" (UID: "41811469-4ed5-4543-ba64-a907100bc569") : secret "kube-state-metrics-tls" not found
Jul 25 00:40:53.972089 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/41811469-4ed5-4543-ba64-a907100bc569-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.972089 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.972493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972388    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-sys\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.972493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972453    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76165ad8-fb19-412c-96f0-bb972c9d72b6-root\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.972659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.972813 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.972788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.974828 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.974502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.975220 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.975199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.975328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.975293    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76165ad8-fb19-412c-96f0-bb972c9d72b6-node-exporter-tls\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:53.979392 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.979347    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xvzh\" (UniqueName: \"kubernetes.io/projected/41811469-4ed5-4543-ba64-a907100bc569-kube-api-access-2xvzh\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:53.979811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:53.979792    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bv5n\" (UniqueName: \"kubernetes.io/projected/76165ad8-fb19-412c-96f0-bb972c9d72b6-kube-api-access-2bv5n\") pod \"node-exporter-dnwsz\" (UID: \"76165ad8-fb19-412c-96f0-bb972c9d72b6\") " pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:54.034811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.034734    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:54.094663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.094639    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnwsz"
Jul 25 00:40:54.129397 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.129360    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:54.384503 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:54.384476    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76165ad8_fb19_412c_96f0_bb972c9d72b6.slice/crio-4d08aefce9eb0a0f6e9c699750e2b576d8b0ca9805deeba04c98be292e3b05a3 WatchSource:0}: Error finding container 4d08aefce9eb0a0f6e9c699750e2b576d8b0ca9805deeba04c98be292e3b05a3: Status 404 returned error can't find the container with id 4d08aefce9eb0a0f6e9c699750e2b576d8b0ca9805deeba04c98be292e3b05a3
Jul 25 00:40:54.386651 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.386632    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:40:54.474160 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.474128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:54.476374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.476344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/41811469-4ed5-4543-ba64-a907100bc569-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v95ws\" (UID: \"41811469-4ed5-4543-ba64-a907100bc569\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:54.579052 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.579013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:54.700981 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.700955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:54.714367 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.714342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"
Jul 25 00:40:54.787288 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.787035    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:54.788390 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.787883    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:40:54.810692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.810641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.813607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.813439    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:40:54.818205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.814941    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 25 00:40:54.818205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.815474    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 25 00:40:54.818205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.815490    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 25 00:40:54.818205 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.815997    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.820948    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.821330    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znw2h\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.822238    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.822258    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.822286    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 25 00:40:54.822555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.822432    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 25 00:40:54.878193 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878126    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878193 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfswt\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878416 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878559    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878636 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.878907 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.878640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.884480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.884367    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws"]
Jul 25 00:40:54.888882 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:54.888852    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41811469_4ed5_4543_ba64_a907100bc569.slice/crio-b659dcfeb4cc4ea062e89d54130b2475c4058c824b768a605c69f2a01178372a WatchSource:0}: Error finding container b659dcfeb4cc4ea062e89d54130b2475c4058c824b768a605c69f2a01178372a: Status 404 returned error can't find the container with id b659dcfeb4cc4ea062e89d54130b2475c4058c824b768a605c69f2a01178372a
Jul 25 00:40:54.943414 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.943284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-rhg4x" event={"ID":"64b7e67f-2fb5-43d9-af0a-23df633b13a3","Type":"ContainerStarted","Data":"7d5e965f0c3f00504b93d1116f0990ec70617dc900bf5702cd4759e94a88fbe3"}
Jul 25 00:40:54.943414 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.943347    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:54.944705 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.944680    2574 patch_prober.go:28] interesting pod/downloads-7c97dc4555-rhg4x container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Jul 25 00:40:54.944801 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.944727    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-rhg4x" podUID="64b7e67f-2fb5-43d9-af0a-23df633b13a3" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused"
Jul 25 00:40:54.945668 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.945638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnwsz" event={"ID":"76165ad8-fb19-412c-96f0-bb972c9d72b6","Type":"ContainerStarted","Data":"4d08aefce9eb0a0f6e9c699750e2b576d8b0ca9805deeba04c98be292e3b05a3"}
Jul 25 00:40:54.948215 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.948190    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws" event={"ID":"41811469-4ed5-4543-ba64-a907100bc569","Type":"ContainerStarted","Data":"b659dcfeb4cc4ea062e89d54130b2475c4058c824b768a605c69f2a01178372a"}
Jul 25 00:40:54.959381 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.959335    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-rhg4x" podStartSLOduration=1.116317344 podStartE2EDuration="18.959318482s" totalImagesPullingTime="17.843001138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:54.957433997 +0000 UTC m=+309.668852395" watchObservedRunningTime="2026-07-25 00:40:54.959318482 +0000 UTC m=+309.670736882"
Jul 25 00:40:54.979244 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979421    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979476    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfswt\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979560    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.979601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.980041 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.979634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.980713 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.980383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.982475 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.981277    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.982475 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:54.982208    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle podName:aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d nodeName:}" failed. No retries permitted until 2026-07-25 00:40:55.482189427 +0000 UTC m=+310.193607817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d") : configmap references non-existent config key: ca-bundle.crt
Jul 25 00:40:54.982475 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.982468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.982719 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:54.982531    2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 25 00:40:54.982719 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:40:54.982599    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls podName:aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d nodeName:}" failed. No retries permitted until 2026-07-25 00:40:55.482583021 +0000 UTC m=+310.194001410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d") : secret "alertmanager-main-tls" not found
Jul 25 00:40:54.983374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.983329    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.985015 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.984960    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.985388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.985348    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.985842 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.985504    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.986118 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.986079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.987363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.987323    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.987500 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.987462    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:54.988689 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:54.988652    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfswt\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.202869 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.202736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:55.483780 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.483736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.483780 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.483783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.484715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.484663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.486920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.486872    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.735704 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.735618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:40:55.968715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.968070    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-rhg4x"
Jul 25 00:40:55.970124 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:55.970022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:56.049555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.049456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:56.077302 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.077161    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:40:56.455074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.454460    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:40:56.554866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.554775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:56.661111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.661069    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:56.705217 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.705134    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"]
Jul 25 00:40:56.725422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.725393    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"]
Jul 25 00:40:56.725830 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.725743    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.730355 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.729708    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 25 00:40:56.730355 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.729923    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 25 00:40:56.739641 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.739224    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:56.739784 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.739652    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 25 00:40:56.740093 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.740074    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mfhvp\""
Jul 25 00:40:56.740386 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.740366    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-74h59atjuj3v\""
Jul 25 00:40:56.741445 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.741419    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 25 00:40:56.741784 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.741745    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.805991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806129    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-grpc-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g6wg\" (UniqueName: \"kubernetes.io/projected/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-kube-api-access-5g6wg\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.806365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.806372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-metrics-client-ca\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.906919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.906967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-grpc-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g6wg\" (UniqueName: \"kubernetes.io/projected/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-kube-api-access-5g6wg\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-metrics-client-ca\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.907553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.907213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.908942 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.908878    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-metrics-client-ca\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.916614 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.916566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g6wg\" (UniqueName: \"kubernetes.io/projected/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-kube-api-access-5g6wg\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.917398 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.917353    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.918996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.918953    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.919544 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.919486    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.920237 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.920198    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.926871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.926827    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.933011 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.932970    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e50a877-856a-4ed5-b4e3-cfea0899f9f6-secret-grpc-tls\") pod \"thanos-querier-69f88cbbd9-p2fgs\" (UID: \"5e50a877-856a-4ed5-b4e3-cfea0899f9f6\") " pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:56.968121 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.967791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"d63bbce8155f7bc7b607325ebd0b8929dba9e7ab096b22e71aa9488478afaf53"}
Jul 25 00:40:56.970742 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.970386    2574 generic.go:356] "Generic (PLEG): container finished" podID="76165ad8-fb19-412c-96f0-bb972c9d72b6" containerID="07e099c2bc88f1706ffa116824bd65b915a45237774f36ea779ce02ab55c34a9" exitCode=0
Jul 25 00:40:56.971592 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:56.971566    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnwsz" event={"ID":"76165ad8-fb19-412c-96f0-bb972c9d72b6","Type":"ContainerDied","Data":"07e099c2bc88f1706ffa116824bd65b915a45237774f36ea779ce02ab55c34a9"}
Jul 25 00:40:57.055420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:57.055335    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:57.070058 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:57.070027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:40:57.129708 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:57.128860    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:57.226343 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:57.226207    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:57.327125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:57.327083    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:58.233802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.233709    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"]
Jul 25 00:40:58.234894 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:40:58.234820    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e50a877_856a_4ed5_b4e3_cfea0899f9f6.slice/crio-c5afed86eb4993c6f49cf79c77e74d6786cec91d67076393f6ceb09701db6acc WatchSource:0}: Error finding container c5afed86eb4993c6f49cf79c77e74d6786cec91d67076393f6ceb09701db6acc: Status 404 returned error can't find the container with id c5afed86eb4993c6f49cf79c77e74d6786cec91d67076393f6ceb09701db6acc
Jul 25 00:40:58.495063 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.495033    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:58.502835 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.502695    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"]
Jul 25 00:40:58.534498 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.534446    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"]
Jul 25 00:40:58.534661 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.534571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.536990 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.536782    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 25 00:40:58.536990 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.536909    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jqr77\""
Jul 25 00:40:58.554943 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.554878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:58.609210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.609176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:40:58.625550 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.625499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb80a9ca-8027-48d1-a26f-af96708b6e79-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.625682 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.625654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wz4\" (UniqueName: \"kubernetes.io/projected/cb80a9ca-8027-48d1-a26f-af96708b6e79-kube-api-access-h5wz4\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.726645 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.726604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wz4\" (UniqueName: \"kubernetes.io/projected/cb80a9ca-8027-48d1-a26f-af96708b6e79-kube-api-access-h5wz4\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.726814 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.726685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb80a9ca-8027-48d1-a26f-af96708b6e79-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.729314 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.729287    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb80a9ca-8027-48d1-a26f-af96708b6e79-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.733802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.733775    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wz4\" (UniqueName: \"kubernetes.io/projected/cb80a9ca-8027-48d1-a26f-af96708b6e79-kube-api-access-h5wz4\") pod \"monitoring-plugin-68b7fc9574-crwnp\" (UID: \"cb80a9ca-8027-48d1-a26f-af96708b6e79\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.849117 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.849035    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:40:58.989725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.989689    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"c5afed86eb4993c6f49cf79c77e74d6786cec91d67076393f6ceb09701db6acc"}
Jul 25 00:40:58.994441 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.994399    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a" exitCode=0
Jul 25 00:40:58.994612 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:58.994528    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a"}
Jul 25 00:40:59.025691 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.025645    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnwsz" event={"ID":"76165ad8-fb19-412c-96f0-bb972c9d72b6","Type":"ContainerStarted","Data":"6d3d9a001e683820dcfac9c34f38cc48c82af9b7748666d5226c1d45a139baef"}
Jul 25 00:40:59.025841 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.025738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnwsz" event={"ID":"76165ad8-fb19-412c-96f0-bb972c9d72b6","Type":"ContainerStarted","Data":"1ea20652a30aaf5497a2210fed88fa85a143b4368cca1a2978ce20acba73da6e"}
Jul 25 00:40:59.034010 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.033861    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws" event={"ID":"41811469-4ed5-4543-ba64-a907100bc569","Type":"ContainerStarted","Data":"bf400b68afcd48445e85e02633bcca489e8889a7292507bf5916d4554dd72a62"}
Jul 25 00:40:59.034010 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.033926    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws" event={"ID":"41811469-4ed5-4543-ba64-a907100bc569","Type":"ContainerStarted","Data":"61c9cf46dce505dcfb5b1f559186aa7653c8ace1b3c52da92164d9d82a255767"}
Jul 25 00:40:59.034010 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.033954    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws" event={"ID":"41811469-4ed5-4543-ba64-a907100bc569","Type":"ContainerStarted","Data":"92cf4e84f88828b89f876df28deaf23dc754605be13fef3afff014c0c1d3a1d0"}
Jul 25 00:40:59.047694 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.047629    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dnwsz" podStartSLOduration=4.449827979 podStartE2EDuration="6.047609487s" totalImagesPullingTime="1.500647742s" totalInitContainerRuntime="97.133766ms" isStatefulPod=true podCreationTimestamp="2026-07-25 00:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:59.045379463 +0000 UTC m=+313.756797887" watchObservedRunningTime="2026-07-25 00:40:59.047609487 +0000 UTC m=+313.759027887"
Jul 25 00:40:59.050178 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.050155    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"]
Jul 25 00:40:59.065845 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.063753    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v95ws" podStartSLOduration=2.887863669 podStartE2EDuration="6.063733948s" totalImagesPullingTime="3.175870279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:40:59.063041889 +0000 UTC m=+313.774460288" watchObservedRunningTime="2026-07-25 00:40:59.063733948 +0000 UTC m=+313.775152348"
Jul 25 00:40:59.899092 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:40:59.898979    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:00.055456 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:00.054867    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp" event={"ID":"cb80a9ca-8027-48d1-a26f-af96708b6e79","Type":"ContainerStarted","Data":"1d6d9c531d7cc2a013af41b932458bf0cf54a5e86aa912f7c3c66dc6d6fcf95d"}
Jul 25 00:41:00.108237 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:00.108115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:02.894614 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.894586    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:41:02.909773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.909746    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:41:02.909880 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.909862    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.913046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.913022    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 25 00:41:02.913274 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.913251    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 25 00:41:02.913483 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.913462    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6m2t\""
Jul 25 00:41:02.913718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.913695    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 25 00:41:02.915208 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.915183    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 25 00:41:02.915953 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.915931    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 25 00:41:02.919181 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.919091    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 25 00:41:02.967527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967644 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967644 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967744 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967667    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967744 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967820 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmpr4\" (UniqueName: \"kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:02.967863 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:02.967822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069118    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmpr4\" (UniqueName: \"kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.069458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.069286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.071494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.070824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.071494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.070982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.071494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.071408    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.071494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.071496    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.072088 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.072044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.073914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.073732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.075538 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.075437    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"db11e61e31cebe1d8928d18e9bbc9b84087e651010f3f0fa8894931ce376c9d7"}
Jul 25 00:41:03.076653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.075726    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"a58ec2f8ede2e722bec08c771a3af0228cc9059e806005cec4fdefd324481ced"}
Jul 25 00:41:03.076653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.075799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"1570359dc8eba51e155d7ff10244a9240d3db19cc58c84e6d611600d5f0c5794"}
Jul 25 00:41:03.083023 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.082737    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"}
Jul 25 00:41:03.083023 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.082826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"}
Jul 25 00:41:03.083023 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.082880    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"}
Jul 25 00:41:03.084442 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.084386    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmpr4\" (UniqueName: \"kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4\") pod \"console-7c77fd57c9-cvzft\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") " pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.085663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.085639    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp" event={"ID":"cb80a9ca-8027-48d1-a26f-af96708b6e79","Type":"ContainerStarted","Data":"ef89567403ae8110a4dbec20b20168feafff6404e08aa2918af88ca681ac5ae7"}
Jul 25 00:41:03.102393 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.102371    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:41:03.108932 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.108913    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp"
Jul 25 00:41:03.117051 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.117010    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-crwnp" podStartSLOduration=1.604247189 podStartE2EDuration="5.116993453s" totalImagesPullingTime="3.512746264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:03.115898937 +0000 UTC m=+317.827317337" watchObservedRunningTime="2026-07-25 00:41:03.116993453 +0000 UTC m=+317.828411854"
Jul 25 00:41:03.225427 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.225399    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:03.399686 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:03.399620    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:41:03.402061 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:41:03.402031    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f44b67_c815_44e0_b489_e4411e84c0be.slice/crio-9e94db0f29243238d532547fa138e1f9c6df40bc42fe181cdf582c9f35b36d95 WatchSource:0}: Error finding container 9e94db0f29243238d532547fa138e1f9c6df40bc42fe181cdf582c9f35b36d95: Status 404 returned error can't find the container with id 9e94db0f29243238d532547fa138e1f9c6df40bc42fe181cdf582c9f35b36d95
Jul 25 00:41:04.093285 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:04.093250    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c77fd57c9-cvzft" event={"ID":"b8f44b67-c815-44e0-b489-e4411e84c0be","Type":"ContainerStarted","Data":"9e94db0f29243238d532547fa138e1f9c6df40bc42fe181cdf582c9f35b36d95"}
Jul 25 00:41:04.096956 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:04.096925    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"}
Jul 25 00:41:04.097090 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:04.097002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"}
Jul 25 00:41:05.104069 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.104034    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"6b6a8bb90059a6b674e39f6ff7bba41899e6ee9f37a85718518eef87ae8960b6"}
Jul 25 00:41:05.104611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.104104    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"c8bb3985ffc33d34647c3cf1a644856c469c0b6eff77e42195772bc0a23b3757"}
Jul 25 00:41:05.104611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.104136    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" event={"ID":"5e50a877-856a-4ed5-b4e3-cfea0899f9f6","Type":"ContainerStarted","Data":"ecd74371c0e85a9c9619b5c4a9ab728e1529212389b96fdd3e8a7013d5fa078e"}
Jul 25 00:41:05.104611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.104199    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:41:05.107191 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.107163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerStarted","Data":"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"}
Jul 25 00:41:05.125738 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.125692    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs" podStartSLOduration=3.522900182 podStartE2EDuration="9.125675186s" totalImagesPullingTime="5.602775004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:05.124412177 +0000 UTC m=+319.835830579" watchObservedRunningTime="2026-07-25 00:41:05.125675186 +0000 UTC m=+319.837093587"
Jul 25 00:41:05.156579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.156528    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.331033177 podStartE2EDuration="11.156497771s" totalImagesPullingTime="6.797359016s" totalInitContainerRuntime="28.105578ms" isStatefulPod=false podCreationTimestamp="2026-07-25 00:40:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:05.155610008 +0000 UTC m=+319.867028406" watchObservedRunningTime="2026-07-25 00:41:05.156497771 +0000 UTC m=+319.867916170"
Jul 25 00:41:05.167578 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.167541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:05.282834 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:05.282794    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:06.566706 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:06.566673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:06.667863 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:06.667831    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:07.121674 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:07.121586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c77fd57c9-cvzft" event={"ID":"b8f44b67-c815-44e0-b489-e4411e84c0be","Type":"ContainerStarted","Data":"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"}
Jul 25 00:41:07.139243 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:07.139194    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c77fd57c9-cvzft" podStartSLOduration=1.809249567 podStartE2EDuration="5.139179663s" totalImagesPullingTime="3.329930096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:41:07.137227154 +0000 UTC m=+321.848645554" watchObservedRunningTime="2026-07-25 00:41:07.139179663 +0000 UTC m=+321.850598059"
Jul 25 00:41:07.272535 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:07.272476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:07.322616 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:07.322581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:09.546038 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:09.546005    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:09.649333 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:09.649291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:11.097358 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:11.097313    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:11.122379 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:11.122351    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-69f88cbbd9-p2fgs"
Jul 25 00:41:11.151784 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:11.151754    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:13.225784 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:13.225746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:13.226259 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:13.225824    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:13.230793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:13.230766    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:14.154580 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:14.154552    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:41:14.310533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:14.310479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:14.406893 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:14.406795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:14.507079 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:14.507023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:14.607431 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:14.607391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:18.655775 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:18.655733    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:18.707250 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:18.707219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:20.155926 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:20.155895    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:20.205714 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:20.205683    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:21.482897 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.482831    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerName="registry" containerID="cri-o://5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223" gracePeriod=30
Jul 25 00:41:21.760384 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.760320    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:41:21.945395 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945364    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945404    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945426    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxjbp\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945443    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945582    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945704    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.945750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.945740    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token\") pod \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\" (UID: \"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5\") "
Jul 25 00:41:21.946176 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.946128    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:41:21.946314 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.946204    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:41:21.948231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.948205    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:41:21.948373 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.948344    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp" (OuterVolumeSpecName: "kube-api-access-gxjbp") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "kube-api-access-gxjbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:41:21.948439 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.948348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:41:21.948481 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.948438    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:41:21.948481 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.948446    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:41:21.954149 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:21.954126    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" (UID: "1dc22a0b-458b-4389-b3ef-f6ccf48c5de5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046815    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-ca-trust-extracted\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046843    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046852    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxjbp\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-kube-api-access-gxjbp\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046861    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-image-registry-private-configuration\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046871    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-registry-certificates\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046879    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-installation-pull-secrets\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046887    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-trusted-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.046892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.046895    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5-bound-sa-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:41:22.184186 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.184156    2574 generic.go:356] "Generic (PLEG): container finished" podID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerID="5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223" exitCode=0
Jul 25 00:41:22.184345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.184196    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n"
Jul 25 00:41:22.184345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.184210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" event={"ID":"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5","Type":"ContainerDied","Data":"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"}
Jul 25 00:41:22.184345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.184249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c5f665fb-xk66n" event={"ID":"1dc22a0b-458b-4389-b3ef-f6ccf48c5de5","Type":"ContainerDied","Data":"36ef243b462067509e6c68f16982b07135aeb93c3662e6a04b35a4aefb4371f2"}
Jul 25 00:41:22.184345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.184269    2574 scope.go:117] "RemoveContainer" containerID="5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"
Jul 25 00:41:22.196287 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.196270    2574 scope.go:117] "RemoveContainer" containerID="5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"
Jul 25 00:41:22.196578 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:41:22.196557    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223\": container with ID starting with 5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223 not found: ID does not exist" containerID="5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"
Jul 25 00:41:22.196634 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.196586    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223"} err="failed to get container status \"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223\": rpc error: code = NotFound desc = could not find container \"5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223\": container with ID starting with 5b508ddfb818da8ed0664f3430f9025dea3c5cbe6572bdd3913deafdc9ee9223 not found: ID does not exist"
Jul 25 00:41:22.204193 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.204173    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:41:22.207945 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:22.207917    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84c5f665fb-xk66n"]
Jul 25 00:41:23.931387 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:23.931353    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" path="/var/lib/kubelet/pods/1dc22a0b-458b-4389-b3ef-f6ccf48c5de5/volumes"
Jul 25 00:41:24.193778 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:24.193704    2574 generic.go:356] "Generic (PLEG): container finished" podID="00d17a42-c88c-4260-87e6-bd0d9a8618e5" containerID="67671f51510641121571203a7554a70da8fc7a5b9854a361f2e3f09c4f9d3b8b" exitCode=0
Jul 25 00:41:24.193778 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:24.193779    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n8sw9" event={"ID":"00d17a42-c88c-4260-87e6-bd0d9a8618e5","Type":"ContainerDied","Data":"67671f51510641121571203a7554a70da8fc7a5b9854a361f2e3f09c4f9d3b8b"}
Jul 25 00:41:24.194136 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:24.194121    2574 scope.go:117] "RemoveContainer" containerID="67671f51510641121571203a7554a70da8fc7a5b9854a361f2e3f09c4f9d3b8b"
Jul 25 00:41:24.199105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:24.199056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:24.295236 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:24.295209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:25.199772 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:25.199740    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-n8sw9" event={"ID":"00d17a42-c88c-4260-87e6-bd0d9a8618e5","Type":"ContainerStarted","Data":"204a010ab1e9de38508b19e9937a39351918041c53172bec2e792c24c224d0f3"}
Jul 25 00:41:32.896034 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:32.895943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:32.945656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:32.945595    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:33.231190 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.231157    2574 generic.go:356] "Generic (PLEG): container finished" podID="b0b9f0f3-700d-4804-9885-87868abc05df" containerID="8320ccfdcf5f4a722d8e6899fb6c3ee722fd56f9e72f22ba360eca0ae9426e66" exitCode=0
Jul 25 00:41:33.231366 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.231224    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6" event={"ID":"b0b9f0f3-700d-4804-9885-87868abc05df","Type":"ContainerDied","Data":"8320ccfdcf5f4a722d8e6899fb6c3ee722fd56f9e72f22ba360eca0ae9426e66"}
Jul 25 00:41:33.231637 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.231615    2574 scope.go:117] "RemoveContainer" containerID="8320ccfdcf5f4a722d8e6899fb6c3ee722fd56f9e72f22ba360eca0ae9426e66"
Jul 25 00:41:33.232855 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.232836    2574 generic.go:356] "Generic (PLEG): container finished" podID="3fe854e2-c0e9-4e36-8d35-ab77a8aee495" containerID="f1c3afb858d93dcf4d630bd49789199fe7e51eb25174eea9637aab1831cb8f6b" exitCode=0
Jul 25 00:41:33.232950 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.232870    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj" event={"ID":"3fe854e2-c0e9-4e36-8d35-ab77a8aee495","Type":"ContainerDied","Data":"f1c3afb858d93dcf4d630bd49789199fe7e51eb25174eea9637aab1831cb8f6b"}
Jul 25 00:41:33.233168 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:33.233152    2574 scope.go:117] "RemoveContainer" containerID="f1c3afb858d93dcf4d630bd49789199fe7e51eb25174eea9637aab1831cb8f6b"
Jul 25 00:41:34.239383 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:34.239349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-dgktj" event={"ID":"3fe854e2-c0e9-4e36-8d35-ab77a8aee495","Type":"ContainerStarted","Data":"fefb9294f16bfdb6ee4ecf54366f3b8fb1dbbdb9a4db779ebe7005a5fbf0fa1b"}
Jul 25 00:41:34.241165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:34.241143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6" event={"ID":"b0b9f0f3-700d-4804-9885-87868abc05df","Type":"ContainerStarted","Data":"237a2f38b35f5585b91e6f8fbde1c946a1f5ab6e0d067eeb01144489fb5d8855"}
Jul 25 00:41:53.880038 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:53.879997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:41:59.688976 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:59.688915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:41:59.691305 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:59.691280    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67c6b28-9884-4498-bd1f-0b5f7d52231f-metrics-certs\") pod \"network-metrics-daemon-wdr58\" (UID: \"c67c6b28-9884-4498-bd1f-0b5f7d52231f\") " pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:41:59.729798 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:59.729768    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7xdtb\""
Jul 25 00:41:59.737736 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:59.737707    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdr58"
Jul 25 00:41:59.863165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:41:59.863135    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdr58"]
Jul 25 00:41:59.866435 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:41:59.866410    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67c6b28_9884_4498_bd1f_0b5f7d52231f.slice/crio-e3876d36039f59030c137674197ed3469e423d4682f30691947df35dc90c6765 WatchSource:0}: Error finding container e3876d36039f59030c137674197ed3469e423d4682f30691947df35dc90c6765: Status 404 returned error can't find the container with id e3876d36039f59030c137674197ed3469e423d4682f30691947df35dc90c6765
Jul 25 00:42:00.356262 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:00.356225    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdr58" event={"ID":"c67c6b28-9884-4498-bd1f-0b5f7d52231f","Type":"ContainerStarted","Data":"e3876d36039f59030c137674197ed3469e423d4682f30691947df35dc90c6765"}
Jul 25 00:42:01.363129 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:01.363092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdr58" event={"ID":"c67c6b28-9884-4498-bd1f-0b5f7d52231f","Type":"ContainerStarted","Data":"79e8385fd719154e1ca9a83e4cc7ebaaf050764c25ab29dd3742a3d9e4b74156"}
Jul 25 00:42:01.363539 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:01.363148    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdr58" event={"ID":"c67c6b28-9884-4498-bd1f-0b5f7d52231f","Type":"ContainerStarted","Data":"c133c12c86f70efb96596643d294e0328646a09c4fd7b65505f6387fc53008ec"}
Jul 25 00:42:01.379076 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:01.379028    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wdr58" podStartSLOduration=373.536017838 podStartE2EDuration="6m14.379011861s" totalImagesPullingTime="842.994023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:42:01.377973123 +0000 UTC m=+376.089391524" watchObservedRunningTime="2026-07-25 00:42:01.379011861 +0000 UTC m=+376.090430261"
Jul 25 00:42:01.465388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:01.465351    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:01.518944 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:01.518911    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:12.903825 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:12.903788    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:13.121063 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.121023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:13.970600 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970568    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:13.970984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970835    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="alertmanager" containerID="cri-o://21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed" gracePeriod=120
Jul 25 00:42:13.970984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970897    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-metric" containerID="cri-o://3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887" gracePeriod=120
Jul 25 00:42:13.970984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970942    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy" containerID="cri-o://dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f" gracePeriod=120
Jul 25 00:42:13.971148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970900    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-web" containerID="cri-o://a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e" gracePeriod=120
Jul 25 00:42:13.971148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.970972    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="config-reloader" containerID="cri-o://6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096" gracePeriod=120
Jul 25 00:42:13.971148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:13.971002    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="prom-label-proxy" containerID="cri-o://f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9" gracePeriod=120
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417760    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9" exitCode=0
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417785    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887" exitCode=0
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417793    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f" exitCode=0
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417799    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096" exitCode=0
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417805    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed" exitCode=0
Jul 25 00:42:14.417837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"}
Jul 25 00:42:14.418115 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417916    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"}
Jul 25 00:42:14.418115 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"}
Jul 25 00:42:14.418115 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.417966    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"}
Jul 25 00:42:14.418115 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:14.418001    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"}
Jul 25 00:42:15.226260 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.226236    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.321410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321321    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321358    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321379    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfswt\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321406    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321423    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321445    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321472    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321521    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321549    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321577    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321623    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321651    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.321770 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321676    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") pod \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\" (UID: \"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d\") "
Jul 25 00:42:15.322209 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.321910    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:15.322209 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.322123    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:42:15.323073 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.323029    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:15.324273 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.324244    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:42:15.324953 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.324919    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt" (OuterVolumeSpecName: "kube-api-access-xfswt") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "kube-api-access-xfswt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:42:15.325071 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325045    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out" (OuterVolumeSpecName: "config-out") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:42:15.325493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325466    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.325659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325592    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.325659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325609    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.325838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325814    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume" (OuterVolumeSpecName: "config-volume") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.325916 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.325837    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.328848 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.328826    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.334991 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.334970    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config" (OuterVolumeSpecName: "web-config") pod "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" (UID: "aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:15.422566 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422534    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422566 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422566    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422582    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-main-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422598    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-tls-assets\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422610    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-out\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422622    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfswt\" (UniqueName: \"kubernetes.io/projected/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-kube-api-access-xfswt\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422631    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-web-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422640    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422648    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-alertmanager-main-db\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422657    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422665    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-metrics-client-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422674    2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.422769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.422684    2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d-cluster-tls-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:15.424251 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.424226    2574 generic.go:356] "Generic (PLEG): container finished" podID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerID="a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e" exitCode=0
Jul 25 00:42:15.424360 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.424294    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.424360 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.424305    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"}
Jul 25 00:42:15.424473 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.424392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d","Type":"ContainerDied","Data":"d63bbce8155f7bc7b607325ebd0b8929dba9e7ab096b22e71aa9488478afaf53"}
Jul 25 00:42:15.424473 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.424446    2574 scope.go:117] "RemoveContainer" containerID="f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"
Jul 25 00:42:15.433376 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.433192    2574 scope.go:117] "RemoveContainer" containerID="3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"
Jul 25 00:42:15.441691 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.441673    2574 scope.go:117] "RemoveContainer" containerID="dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"
Jul 25 00:42:15.447759 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.447736    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:15.449010 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.448988    2574 scope.go:117] "RemoveContainer" containerID="a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"
Jul 25 00:42:15.454263 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.454240    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:15.456459 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.456443    2574 scope.go:117] "RemoveContainer" containerID="6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"
Jul 25 00:42:15.463351 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.463334    2574 scope.go:117] "RemoveContainer" containerID="21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"
Jul 25 00:42:15.470502 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.470486    2574 scope.go:117] "RemoveContainer" containerID="e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a"
Jul 25 00:42:15.478089 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478073    2574 scope.go:117] "RemoveContainer" containerID="f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"
Jul 25 00:42:15.478155 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478092    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:15.478370 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.478341    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9\": container with ID starting with f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9 not found: ID does not exist" containerID="f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"
Jul 25 00:42:15.478422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478381    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9"} err="failed to get container status \"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9\": rpc error: code = NotFound desc = could not find container \"f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9\": container with ID starting with f40482574d1b7dec4defa7f9ce39315992d1fd5e114f4d29d50e10dc695985b9 not found: ID does not exist"
Jul 25 00:42:15.478422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478400    2574 scope.go:117] "RemoveContainer" containerID="3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478427    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="alertmanager"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478438    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="alertmanager"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478448    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-web"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478455    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-web"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478464    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="init-config-reloader"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478473    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="init-config-reloader"
Jul 25 00:42:15.478494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478492    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerName="registry"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478501    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerName="registry"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478528    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="config-reloader"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478538    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="config-reloader"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478550    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478558    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478567    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-metric"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478575    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-metric"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478584    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="prom-label-proxy"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478592    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="prom-label-proxy"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478664    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc22a0b-458b-4389-b3ef-f6ccf48c5de5" containerName="registry"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478673    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-metric"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478679    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy-web"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478687    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="alertmanager"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478693    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="config-reloader"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.478683    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887\": container with ID starting with 3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887 not found: ID does not exist" containerID="3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478714    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887"} err="failed to get container status \"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887\": rpc error: code = NotFound desc = could not find container \"3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887\": container with ID starting with 3037978c892fb7c5de417e62a11d99a4c45ca7e7e0f3fcd5cb00cea123840887 not found: ID does not exist"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478736    2574 scope.go:117] "RemoveContainer" containerID="dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478699    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="prom-label-proxy"
Jul 25 00:42:15.478793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.478773    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" containerName="kube-rbac-proxy"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.479036    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f\": container with ID starting with dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f not found: ID does not exist" containerID="dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479061    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f"} err="failed to get container status \"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f\": rpc error: code = NotFound desc = could not find container \"dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f\": container with ID starting with dad8cb9bf116c11ce5a165aa8f1c75955fc073105d27678947a6b9fe2ac8632f not found: ID does not exist"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479082    2574 scope.go:117] "RemoveContainer" containerID="a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.479373    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e\": container with ID starting with a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e not found: ID does not exist" containerID="a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479396    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e"} err="failed to get container status \"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e\": rpc error: code = NotFound desc = could not find container \"a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e\": container with ID starting with a3fde608b326c1ee112bf1a7f8c65ea8a53f8efdf4035ab7975ec0fbc437382e not found: ID does not exist"
Jul 25 00:42:15.479562 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479412    2574 scope.go:117] "RemoveContainer" containerID="6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"
Jul 25 00:42:15.479766 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.479664    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096\": container with ID starting with 6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096 not found: ID does not exist" containerID="6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"
Jul 25 00:42:15.479766 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479689    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096"} err="failed to get container status \"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096\": rpc error: code = NotFound desc = could not find container \"6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096\": container with ID starting with 6f3c72947cad3005c57ca1df5bce933656bd7368a9386b98ca78c347cf7df096 not found: ID does not exist"
Jul 25 00:42:15.479766 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479702    2574 scope.go:117] "RemoveContainer" containerID="21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"
Jul 25 00:42:15.479947 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.479930    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed\": container with ID starting with 21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed not found: ID does not exist" containerID="21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"
Jul 25 00:42:15.479986 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479951    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed"} err="failed to get container status \"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed\": rpc error: code = NotFound desc = could not find container \"21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed\": container with ID starting with 21ad84e56d39422100a1b564bc8eb0379ff6aa4a43d7fcda74c26754827a40ed not found: ID does not exist"
Jul 25 00:42:15.479986 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.479965    2574 scope.go:117] "RemoveContainer" containerID="e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a"
Jul 25 00:42:15.480183 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:15.480168    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a\": container with ID starting with e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a not found: ID does not exist" containerID="e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a"
Jul 25 00:42:15.480224 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.480185    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a"} err="failed to get container status \"e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a\": rpc error: code = NotFound desc = could not find container \"e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a\": container with ID starting with e9984db8d25a6d1c7dd42fd44b582234e9a8da708253d7abed4c84acf83afa8a not found: ID does not exist"
Jul 25 00:42:15.483997 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.483980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.498478 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.498459    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 25 00:42:15.498583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.498460    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 25 00:42:15.498583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.498459    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 25 00:42:15.498826 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.498812    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 25 00:42:15.498881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.498843    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znw2h\""
Jul 25 00:42:15.499014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.499002    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 25 00:42:15.499255 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.499242    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 25 00:42:15.499560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.499547    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 25 00:42:15.501630 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.501614    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 25 00:42:15.505701 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.505677    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:15.508703 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.508685    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 25 00:42:15.522968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.522948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523058 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.522976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523058 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.522994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523058 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgjqr\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-kube-api-access-rgjqr\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523159 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523159 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523149    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523221 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-web-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523221 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523204    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-volume\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523223    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-out\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523244    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523269    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.523409 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.523309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623765 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623693    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623765 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-web-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623777    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-volume\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623807    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-out\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623859    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.623955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.623884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624201 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624201 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624201 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624148    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624201 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624394 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgjqr\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-kube-api-access-rgjqr\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624394 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.624926 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.625046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.624927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.626941 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.626912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-volume\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627051 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627260 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627238    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4464bbd-7553-4f3c-8eab-f2b858f46b28-config-out\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627411    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4464bbd-7553-4f3c-8eab-f2b858f46b28-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627840    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-web-config\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.627873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.627847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.628839 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.628822    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.629162 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.629146    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4464bbd-7553-4f3c-8eab-f2b858f46b28-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.633832 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.633814    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgjqr\" (UniqueName: \"kubernetes.io/projected/d4464bbd-7553-4f3c-8eab-f2b858f46b28-kube-api-access-rgjqr\") pod \"alertmanager-main-0\" (UID: \"d4464bbd-7553-4f3c-8eab-f2b858f46b28\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.792411 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.792378    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 25 00:42:15.899808 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.899715    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:15.932409 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:42:15.932380    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4464bbd_7553_4f3c_8eab_f2b858f46b28.slice/crio-c044affd772e5cd129e8bc274185d654a19190ce283b63cd6f9ea1681090157c WatchSource:0}: Error finding container c044affd772e5cd129e8bc274185d654a19190ce283b63cd6f9ea1681090157c: Status 404 returned error can't find the container with id c044affd772e5cd129e8bc274185d654a19190ce283b63cd6f9ea1681090157c
Jul 25 00:42:15.936799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.934347    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d" path="/var/lib/kubelet/pods/aeca6e06-1ed3-48d2-a5d9-0e49c3afd18d/volumes"
Jul 25 00:42:15.936799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:15.935079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 25 00:42:16.000899 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:16.000870    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:16.430831 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:16.430794    2574 generic.go:356] "Generic (PLEG): container finished" podID="d4464bbd-7553-4f3c-8eab-f2b858f46b28" containerID="03716765810a8c783c44bb41d14d5c63d01e2a4d3d35271cf058c7f40a61f22d" exitCode=0
Jul 25 00:42:16.431306 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:16.430876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerDied","Data":"03716765810a8c783c44bb41d14d5c63d01e2a4d3d35271cf058c7f40a61f22d"}
Jul 25 00:42:16.431306 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:16.430950    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"c044affd772e5cd129e8bc274185d654a19190ce283b63cd6f9ea1681090157c"}
Jul 25 00:42:17.441481 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"d09637c59cc0fa4d882b6b221770f92f014360763916adf865c4844c78c55c2d"}
Jul 25 00:42:17.441876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"465bca9384fd00603383a724de958bcb94910c05e8b43570f3fd2a50d095ea41"}
Jul 25 00:42:17.441876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"54cb71359a68facfa549d010e6fc920ff0e5be12d429fd537d4b12c073111caa"}
Jul 25 00:42:17.441876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441625    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"061e965f400acda51285d2de36a51b7c474ab1ba93c463badaa6f95d3b6b1460"}
Jul 25 00:42:17.441876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"5e28cac7084d6b95d657112c01269f86124908bfb1eae8f6213af8a600a225fa"}
Jul 25 00:42:17.441876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.441675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d4464bbd-7553-4f3c-8eab-f2b858f46b28","Type":"ContainerStarted","Data":"1461a5488b6240c75652aafc084c9ac3a4fbd84842cde44f44fccf0645efc92b"}
Jul 25 00:42:17.468455 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.468409    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.449717603 podStartE2EDuration="2.468394089s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.676486ms" isStatefulPod=false podCreationTimestamp="2026-07-25 00:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:42:17.467163578 +0000 UTC m=+392.178581978" watchObservedRunningTime="2026-07-25 00:42:17.468394089 +0000 UTC m=+392.179812489"
Jul 25 00:42:17.983211 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.983175    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-654b776f58-5thfh"]
Jul 25 00:42:17.986915 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.986891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:17.990015 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.989993    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 25 00:42:17.990015 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.990004    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-75chd\""
Jul 25 00:42:17.990150 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.989993    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 25 00:42:17.990328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.990311    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 25 00:42:17.990392 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.990331    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 25 00:42:17.990428 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:17.990320    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 25 00:42:18.001479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.001443    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-654b776f58-5thfh"]
Jul 25 00:42:18.001834 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.001814    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 25 00:42:18.051579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-metrics-client-ca\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtfcx\" (UniqueName: \"kubernetes.io/projected/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-kube-api-access-jtfcx\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-serving-certs-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.051829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.052020 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.051867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-federate-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152405 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-serving-certs-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152405 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-federate-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152629    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-metrics-client-ca\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.152983 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.152778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtfcx\" (UniqueName: \"kubernetes.io/projected/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-kube-api-access-jtfcx\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.153239 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.153208    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-serving-certs-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.153404 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.153379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-metrics-client-ca\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.153566 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.153538    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.155188 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.155165    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-telemeter-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.155286 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.155195    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.155441 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.155423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.155561 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.155541    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-federate-client-tls\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.161302 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.161279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtfcx\" (UniqueName: \"kubernetes.io/projected/3e2896ae-8597-4f8c-a38d-02372ea1f6e4-kube-api-access-jtfcx\") pod \"telemeter-client-654b776f58-5thfh\" (UID: \"3e2896ae-8597-4f8c-a38d-02372ea1f6e4\") " pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.303353 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.303262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh"
Jul 25 00:42:18.434831 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.434808    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-654b776f58-5thfh"]
Jul 25 00:42:18.437351 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:42:18.437316    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2896ae_8597_4f8c_a38d_02372ea1f6e4.slice/crio-3cd1d540930ca7f18104b3c7df6de2709ae58154cc28d2d00ea614cdddf8fae3 WatchSource:0}: Error finding container 3cd1d540930ca7f18104b3c7df6de2709ae58154cc28d2d00ea614cdddf8fae3: Status 404 returned error can't find the container with id 3cd1d540930ca7f18104b3c7df6de2709ae58154cc28d2d00ea614cdddf8fae3
Jul 25 00:42:18.447022 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:18.446996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh" event={"ID":"3e2896ae-8597-4f8c-a38d-02372ea1f6e4","Type":"ContainerStarted","Data":"3cd1d540930ca7f18104b3c7df6de2709ae58154cc28d2d00ea614cdddf8fae3"}
Jul 25 00:42:20.459579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:20.459541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh" event={"ID":"3e2896ae-8597-4f8c-a38d-02372ea1f6e4","Type":"ContainerStarted","Data":"93396cb558bab07c926d33243f7b1b5537a84c1fcd4db5907bcdef364513cbae"}
Jul 25 00:42:20.459954 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:20.459611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh" event={"ID":"3e2896ae-8597-4f8c-a38d-02372ea1f6e4","Type":"ContainerStarted","Data":"9ba8f26f8cc288681da135a128189a9019000d6bb28cdf36125d5005af1dab24"}
Jul 25 00:42:20.459954 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:20.459637    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh" event={"ID":"3e2896ae-8597-4f8c-a38d-02372ea1f6e4","Type":"ContainerStarted","Data":"2bc975aad8edb467780eaa7c79941762e105021b4e98b647f4bff21c31de7075"}
Jul 25 00:42:20.481405 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:20.481357    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-654b776f58-5thfh" podStartSLOduration=1.841709692 podStartE2EDuration="3.481342426s" totalImagesPullingTime="1.639632734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:42:20.479866081 +0000 UTC m=+395.191284480" watchObservedRunningTime="2026-07-25 00:42:20.481342426 +0000 UTC m=+395.192760825"
Jul 25 00:42:21.484649 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.484618    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:42:21.488064 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.488041    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.497693 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.497667    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:42:21.579726 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579684    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.579726 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.579960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.579960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579827    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.579960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g55cc\" (UniqueName: \"kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.580094 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.579970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.580094 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.580002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.600142 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.600104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:21.680612 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680612 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g55cc\" (UniqueName: \"kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.680829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.680768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.681504 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.681478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.681681 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.681534    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.681681 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.681490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.681681 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.681614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.683158 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.683140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.683294 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.683256    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.688080 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.688061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g55cc\" (UniqueName: \"kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc\") pod \"console-75ddd895bb-48x7f\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") " pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.700329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.700299    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:21.748876 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.748795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:21.801063 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.801045    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:21.934028 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:21.933978    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:42:21.936235 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:42:21.936211    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0222a40_004a_4ae9_aa91_07ed0260197e.slice/crio-40b62d5b169452c2bc18835b9b19e64a5ea24f9736a18dc98ef0a0ce420b93bb WatchSource:0}: Error finding container 40b62d5b169452c2bc18835b9b19e64a5ea24f9736a18dc98ef0a0ce420b93bb: Status 404 returned error can't find the container with id 40b62d5b169452c2bc18835b9b19e64a5ea24f9736a18dc98ef0a0ce420b93bb
Jul 25 00:42:22.284028 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:22.283985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:22.333482 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:22.333444    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:22.470776 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:22.470738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ddd895bb-48x7f" event={"ID":"c0222a40-004a-4ae9-aa91-07ed0260197e","Type":"ContainerStarted","Data":"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"}
Jul 25 00:42:22.470947 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:22.470794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ddd895bb-48x7f" event={"ID":"c0222a40-004a-4ae9-aa91-07ed0260197e","Type":"ContainerStarted","Data":"40b62d5b169452c2bc18835b9b19e64a5ea24f9736a18dc98ef0a0ce420b93bb"}
Jul 25 00:42:22.487552 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:22.487479    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75ddd895bb-48x7f" podStartSLOduration=1.487462602 podStartE2EDuration="1.487462602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:42:22.487008778 +0000 UTC m=+397.198427181" watchObservedRunningTime="2026-07-25 00:42:22.487462602 +0000 UTC m=+397.198881000"
Jul 25 00:42:31.801388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:31.801338    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:31.801886 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:31.801434    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:31.806401 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:31.806381    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:32.522011 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:32.521983    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:42:32.564273 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:32.564241    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:42:32.663952 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:32.663921    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:32.714855 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:32.714821    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:32.766946 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:32.766913    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:55.248523 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:55.248480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:55.348156 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:55.348130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:42:57.587332 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.587293    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c77fd57c9-cvzft" podUID="b8f44b67-c815-44e0-b489-e4411e84c0be" containerName="console" containerID="cri-o://0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113" gracePeriod=15
Jul 25 00:42:57.833053 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.833027    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:42:57.994953 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.994929    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.994965    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmpr4\" (UniqueName: \"kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995022    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995044    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995069    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995095    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995384 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995159    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert\") pod \"b8f44b67-c815-44e0-b489-e4411e84c0be\" (UID: \"b8f44b67-c815-44e0-b489-e4411e84c0be\") "
Jul 25 00:42:57.995560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995506    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config" (OuterVolumeSpecName: "console-config") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:57.995695 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995559    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:57.995695 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995654    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:57.995797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.995761    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:42:57.997247 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.997224    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:57.997552 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.997525    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:42:57.997646 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:57.997547    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4" (OuterVolumeSpecName: "kube-api-access-xmpr4") pod "b8f44b67-c815-44e0-b489-e4411e84c0be" (UID: "b8f44b67-c815-44e0-b489-e4411e84c0be"). InnerVolumeSpecName "kube-api-access-xmpr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:42:58.096483 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096452    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmpr4\" (UniqueName: \"kubernetes.io/projected/b8f44b67-c815-44e0-b489-e4411e84c0be-kube-api-access-xmpr4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096483 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096482    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-trusted-ca-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096483 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096492    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-console-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096500    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096526    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096534    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f44b67-c815-44e0-b489-e4411e84c0be-oauth-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.096659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.096542    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f44b67-c815-44e0-b489-e4411e84c0be-console-oauth-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:42:58.631418 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.631386    2574 generic.go:356] "Generic (PLEG): container finished" podID="b8f44b67-c815-44e0-b489-e4411e84c0be" containerID="0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113" exitCode=0
Jul 25 00:42:58.631805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.631435    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c77fd57c9-cvzft"
Jul 25 00:42:58.631805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.631469    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c77fd57c9-cvzft" event={"ID":"b8f44b67-c815-44e0-b489-e4411e84c0be","Type":"ContainerDied","Data":"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"}
Jul 25 00:42:58.631805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.631532    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c77fd57c9-cvzft" event={"ID":"b8f44b67-c815-44e0-b489-e4411e84c0be","Type":"ContainerDied","Data":"9e94db0f29243238d532547fa138e1f9c6df40bc42fe181cdf582c9f35b36d95"}
Jul 25 00:42:58.631805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.631557    2574 scope.go:117] "RemoveContainer" containerID="0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"
Jul 25 00:42:58.640537 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.640500    2574 scope.go:117] "RemoveContainer" containerID="0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"
Jul 25 00:42:58.640789 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:42:58.640771    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113\": container with ID starting with 0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113 not found: ID does not exist" containerID="0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"
Jul 25 00:42:58.640841 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.640797    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113"} err="failed to get container status \"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113\": rpc error: code = NotFound desc = could not find container \"0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113\": container with ID starting with 0fba8817828a6c57dde2c19d19dc5ce0f3ffae06a32d13956ea894ae031fa113 not found: ID does not exist"
Jul 25 00:42:58.652073 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.652048    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:42:58.653464 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:58.653445    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c77fd57c9-cvzft"]
Jul 25 00:42:59.934831 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:42:59.934796    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8f44b67-c815-44e0-b489-e4411e84c0be" path="/var/lib/kubelet/pods/b8f44b67-c815-44e0-b489-e4411e84c0be/volumes"
Jul 25 00:43:28.120982 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:28.120906    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:28.179212 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:28.179176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:30.458867 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.458828    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:43:30.459265 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.459199    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8f44b67-c815-44e0-b489-e4411e84c0be" containerName="console"
Jul 25 00:43:30.459265 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.459213    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8f44b67-c815-44e0-b489-e4411e84c0be" containerName="console"
Jul 25 00:43:30.459395 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.459289    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8f44b67-c815-44e0-b489-e4411e84c0be" containerName="console"
Jul 25 00:43:30.463371 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.463350    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.472396 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.472373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:43:30.511395 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.511355    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:30.552578 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552546    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmfh7\" (UniqueName: \"kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.552829 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.552775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.570896 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.570869    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:30.653944 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.653915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmfh7\" (UniqueName: \"kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.653952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.653970    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654223 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654223 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654338 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654338 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654774 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654750    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654896 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.654896 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.654830    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.655185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.655160    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.656666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.656642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.656860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.656839    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.661058 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.661037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmfh7\" (UniqueName: \"kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7\") pod \"console-bbbf654fd-mdw7x\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") " pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.774709 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.774633    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:30.903687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:30.903663    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:43:30.905959 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:43:30.905929    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796d64eb_47da_4335_9055_072f00bf940c.slice/crio-9627339fa680ee3c1696d94db26acdef622d62b8292f71e5c8531a65d6b73b04 WatchSource:0}: Error finding container 9627339fa680ee3c1696d94db26acdef622d62b8292f71e5c8531a65d6b73b04: Status 404 returned error can't find the container with id 9627339fa680ee3c1696d94db26acdef622d62b8292f71e5c8531a65d6b73b04
Jul 25 00:43:31.781806 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:31.781759    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbf654fd-mdw7x" event={"ID":"796d64eb-47da-4335-9055-072f00bf940c","Type":"ContainerStarted","Data":"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"}
Jul 25 00:43:31.782165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:31.781823    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbf654fd-mdw7x" event={"ID":"796d64eb-47da-4335-9055-072f00bf940c","Type":"ContainerStarted","Data":"9627339fa680ee3c1696d94db26acdef622d62b8292f71e5c8531a65d6b73b04"}
Jul 25 00:43:31.799086 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:31.799041    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbbf654fd-mdw7x" podStartSLOduration=1.799026971 podStartE2EDuration="1.799026971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:43:31.797503556 +0000 UTC m=+466.508921956" watchObservedRunningTime="2026-07-25 00:43:31.799026971 +0000 UTC m=+466.510445370"
Jul 25 00:43:40.021894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.021856    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:40.122762 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.122716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:40.775725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.775690    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:40.775904 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.775791    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:40.780532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.780494    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:40.823671 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.823649    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:43:40.865873 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.865845    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:43:40.965811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:40.965778    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:41.013504 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:41.013481    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:41.065966 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:41.065897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:50.171984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:50.171933    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:50.272423 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:50.272389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:43:50.521105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:43:50.521069    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:05.889427 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:05.889390    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75ddd895bb-48x7f" podUID="c0222a40-004a-4ae9-aa91-07ed0260197e" containerName="console" containerID="cri-o://d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f" gracePeriod=15
Jul 25 00:44:06.132432 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.132405    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:44:06.150250 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150188    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150397 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150261    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150465 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150421    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150536 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150488    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150607 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150589    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g55cc\" (UniqueName: \"kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150620    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:44:06.150663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150627    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:44:06.150663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150637    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150809 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150667    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert\") pod \"c0222a40-004a-4ae9-aa91-07ed0260197e\" (UID: \"c0222a40-004a-4ae9-aa91-07ed0260197e\") "
Jul 25 00:44:06.150855 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150812    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config" (OuterVolumeSpecName: "console-config") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:44:06.150948 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150933    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.151003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150954    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-trusted-ca-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.151003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.150968    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-console-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.151112 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.151084    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:44:06.153231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.153205    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:44:06.153355 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.153238    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:44:06.153666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.153644    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc" (OuterVolumeSpecName: "kube-api-access-g55cc") pod "c0222a40-004a-4ae9-aa91-07ed0260197e" (UID: "c0222a40-004a-4ae9-aa91-07ed0260197e"). InnerVolumeSpecName "kube-api-access-g55cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:44:06.251824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.251796    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g55cc\" (UniqueName: \"kubernetes.io/projected/c0222a40-004a-4ae9-aa91-07ed0260197e-kube-api-access-g55cc\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.251824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.251820    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-oauth-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.251824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.251829    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0222a40-004a-4ae9-aa91-07ed0260197e-oauth-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.252011 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.251838    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0222a40-004a-4ae9-aa91-07ed0260197e-console-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:06.930886 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.930850    2574 generic.go:356] "Generic (PLEG): container finished" podID="c0222a40-004a-4ae9-aa91-07ed0260197e" containerID="d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f" exitCode=0
Jul 25 00:44:06.931270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.930904    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ddd895bb-48x7f"
Jul 25 00:44:06.931270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.930933    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ddd895bb-48x7f" event={"ID":"c0222a40-004a-4ae9-aa91-07ed0260197e","Type":"ContainerDied","Data":"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"}
Jul 25 00:44:06.931270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.930982    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ddd895bb-48x7f" event={"ID":"c0222a40-004a-4ae9-aa91-07ed0260197e","Type":"ContainerDied","Data":"40b62d5b169452c2bc18835b9b19e64a5ea24f9736a18dc98ef0a0ce420b93bb"}
Jul 25 00:44:06.931270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.931004    2574 scope.go:117] "RemoveContainer" containerID="d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"
Jul 25 00:44:06.940074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.940057    2574 scope.go:117] "RemoveContainer" containerID="d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"
Jul 25 00:44:06.940315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:44:06.940295    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f\": container with ID starting with d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f not found: ID does not exist" containerID="d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"
Jul 25 00:44:06.940361 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.940323    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f"} err="failed to get container status \"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f\": rpc error: code = NotFound desc = could not find container \"d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f\": container with ID starting with d924e4bdcb76fe6c0a5f9ac725784f26393a6357b22e1784d8e6452df335137f not found: ID does not exist"
Jul 25 00:44:06.950830 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.950805    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:44:06.954326 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:06.954308    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75ddd895bb-48x7f"]
Jul 25 00:44:07.931457 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:07.931426    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0222a40-004a-4ae9-aa91-07ed0260197e" path="/var/lib/kubelet/pods/c0222a40-004a-4ae9-aa91-07ed0260197e/volumes"
Jul 25 00:44:21.510526 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:21.510478    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:21.608857 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:21.608823    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:22.059579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.059546    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"]
Jul 25 00:44:22.059891 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.059879    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0222a40-004a-4ae9-aa91-07ed0260197e" containerName="console"
Jul 25 00:44:22.059936 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.059893    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0222a40-004a-4ae9-aa91-07ed0260197e" containerName="console"
Jul 25 00:44:22.059970 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.059946    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0222a40-004a-4ae9-aa91-07ed0260197e" containerName="console"
Jul 25 00:44:22.063117 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.063095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.065427 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.065407    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 25 00:44:22.066182 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.066163    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 25 00:44:22.066303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.066166    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b6l4x\""
Jul 25 00:44:22.070380 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.070359    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"]
Jul 25 00:44:22.180790 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.180756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcpzz\" (UniqueName: \"kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.180931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.180847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.180931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.180884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.281583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.281554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcpzz\" (UniqueName: \"kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.281710 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.281633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.281710 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.281661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.282002 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.281982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.282038 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.282004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.289328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.289306    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcpzz\" (UniqueName: \"kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.372444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.372384    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:22.499048 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:22.499026    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"]
Jul 25 00:44:22.501721 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:44:22.501693    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759e9f5f_0f09_48d4_8e2b_64f4fc381551.slice/crio-eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3 WatchSource:0}: Error finding container eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3: Status 404 returned error can't find the container with id eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3
Jul 25 00:44:23.008723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:23.008682    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4" event={"ID":"759e9f5f-0f09-48d4-8e2b-64f4fc381551","Type":"ContainerStarted","Data":"eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3"}
Jul 25 00:44:29.037319 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:29.037281    2574 generic.go:356] "Generic (PLEG): container finished" podID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerID="52fe21bfcbe93f25b14805580f569c43a03385bbe580059f19cf5bc4d2935684" exitCode=0
Jul 25 00:44:29.037745 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:29.037374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4" event={"ID":"759e9f5f-0f09-48d4-8e2b-64f4fc381551","Type":"ContainerDied","Data":"52fe21bfcbe93f25b14805580f569c43a03385bbe580059f19cf5bc4d2935684"}
Jul 25 00:44:32.052027 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:32.051994    2574 generic.go:356] "Generic (PLEG): container finished" podID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerID="7afc5f6917c521876d45aa7648bf1143cccd826caacdb1578809df0a37c89d78" exitCode=0
Jul 25 00:44:32.052407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:32.052087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4" event={"ID":"759e9f5f-0f09-48d4-8e2b-64f4fc381551","Type":"ContainerDied","Data":"7afc5f6917c521876d45aa7648bf1143cccd826caacdb1578809df0a37c89d78"}
Jul 25 00:44:39.091135 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:39.091049    2574 generic.go:356] "Generic (PLEG): container finished" podID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerID="bc6903c4dc898c8e7f7f52dc20197742a755d23319beca1b8b4802791bd5e1d0" exitCode=0
Jul 25 00:44:39.091548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:39.091138    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4" event={"ID":"759e9f5f-0f09-48d4-8e2b-64f4fc381551","Type":"ContainerDied","Data":"bc6903c4dc898c8e7f7f52dc20197742a755d23319beca1b8b4802791bd5e1d0"}
Jul 25 00:44:40.221496 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.221472    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:40.346091 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.346002    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle\") pod \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") "
Jul 25 00:44:40.346091 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.346059    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util\") pod \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") "
Jul 25 00:44:40.346091 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.346079    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcpzz\" (UniqueName: \"kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz\") pod \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\" (UID: \"759e9f5f-0f09-48d4-8e2b-64f4fc381551\") "
Jul 25 00:44:40.346797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.346766    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle" (OuterVolumeSpecName: "bundle") pod "759e9f5f-0f09-48d4-8e2b-64f4fc381551" (UID: "759e9f5f-0f09-48d4-8e2b-64f4fc381551"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:44:40.348268 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.348248    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz" (OuterVolumeSpecName: "kube-api-access-mcpzz") pod "759e9f5f-0f09-48d4-8e2b-64f4fc381551" (UID: "759e9f5f-0f09-48d4-8e2b-64f4fc381551"). InnerVolumeSpecName "kube-api-access-mcpzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:44:40.351322 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.351298    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util" (OuterVolumeSpecName: "util") pod "759e9f5f-0f09-48d4-8e2b-64f4fc381551" (UID: "759e9f5f-0f09-48d4-8e2b-64f4fc381551"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:44:40.447299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.447259    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:40.447299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.447291    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcpzz\" (UniqueName: \"kubernetes.io/projected/759e9f5f-0f09-48d4-8e2b-64f4fc381551-kube-api-access-mcpzz\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:40.447299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:40.447303    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/759e9f5f-0f09-48d4-8e2b-64f4fc381551-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:41.101068 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:41.101032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4" event={"ID":"759e9f5f-0f09-48d4-8e2b-64f4fc381551","Type":"ContainerDied","Data":"eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3"}
Jul 25 00:44:41.101231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:41.101080    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mrnw4"
Jul 25 00:44:41.101231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:41.101088    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb1fb92853cb82bf11abd679e7c08e4973b9c9a76f51092bf65bd3cd8c9adeb3"
Jul 25 00:44:43.018833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:43.018805    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:43.817771 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:43.817729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:44.943269 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943236    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"]
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943567    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="extract"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943578    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="extract"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943590    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="pull"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943595    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="pull"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943614    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="util"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943620    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="util"
Jul 25 00:44:44.943688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.943675    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="759e9f5f-0f09-48d4-8e2b-64f4fc381551" containerName="extract"
Jul 25 00:44:44.987174 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.987146    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"]
Jul 25 00:44:44.987341 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.987237    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:44.989722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.989696    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 25 00:44:44.989722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.989709    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 25 00:44:44.989904 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:44.989741    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vpt5l\""
Jul 25 00:44:45.086499 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.086462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19cc38d5-2241-4a01-8657-f72807b6aa13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.086699 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.086542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87fzc\" (UniqueName: \"kubernetes.io/projected/19cc38d5-2241-4a01-8657-f72807b6aa13-kube-api-access-87fzc\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.187977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.187934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19cc38d5-2241-4a01-8657-f72807b6aa13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.187977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.187978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87fzc\" (UniqueName: \"kubernetes.io/projected/19cc38d5-2241-4a01-8657-f72807b6aa13-kube-api-access-87fzc\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.188312 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.188292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19cc38d5-2241-4a01-8657-f72807b6aa13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.195107 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.195047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87fzc\" (UniqueName: \"kubernetes.io/projected/19cc38d5-2241-4a01-8657-f72807b6aa13-kube-api-access-87fzc\") pod \"cert-manager-operator-controller-manager-55c444bf86-jnh74\" (UID: \"19cc38d5-2241-4a01-8657-f72807b6aa13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.296081 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.296046    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"
Jul 25 00:44:45.448800 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:45.448728    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74"]
Jul 25 00:44:45.452536 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:44:45.452487    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19cc38d5_2241_4a01_8657_f72807b6aa13.slice/crio-300925f0de89e886c1f3f16ce231e74c39186af20d97f727793cee405d9697af WatchSource:0}: Error finding container 300925f0de89e886c1f3f16ce231e74c39186af20d97f727793cee405d9697af: Status 404 returned error can't find the container with id 300925f0de89e886c1f3f16ce231e74c39186af20d97f727793cee405d9697af
Jul 25 00:44:46.123274 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:46.123188    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74" event={"ID":"19cc38d5-2241-4a01-8657-f72807b6aa13","Type":"ContainerStarted","Data":"300925f0de89e886c1f3f16ce231e74c39186af20d97f727793cee405d9697af"}
Jul 25 00:44:46.629105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:46.629065    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:46.683849 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:46.683809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:48.135299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.135257    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74" event={"ID":"19cc38d5-2241-4a01-8657-f72807b6aa13","Type":"ContainerStarted","Data":"f24d89883ae9b10307157c53a0533c47babec8a5065f3293472221daff19005c"}
Jul 25 00:44:48.154802 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.154739    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jnh74" podStartSLOduration=1.980658214 podStartE2EDuration="4.154718915s" totalImagesPullingTime="2.174060701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:44:48.15391241 +0000 UTC m=+542.865330810" watchObservedRunningTime="2026-07-25 00:44:48.154718915 +0000 UTC m=+542.866137317"
Jul 25 00:44:48.231291 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.231259    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:48.331656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.331621    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:48.519994 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.519958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:48.707996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:48.707957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:49.259522 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.259474    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"]
Jul 25 00:44:49.284360 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.284335    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"]
Jul 25 00:44:49.284499 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.284433    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.286833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.286808    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 25 00:44:49.286953 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.286882    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 25 00:44:49.287724 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.287704    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b6l4x\""
Jul 25 00:44:49.425305 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.425265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.425305 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.425308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chtxp\" (UniqueName: \"kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.425533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.425348    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.526721 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.526633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.526721 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.526673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chtxp\" (UniqueName: \"kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.526721 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.526703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.527053 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.527035    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.527109 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.527089    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.537959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.537914    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chtxp\" (UniqueName: \"kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.593721 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.593685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:49.742849 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:49.742813    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"]
Jul 25 00:44:49.746703 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:44:49.746676    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6cfbee_7f43_48d2_b2e7_7807068245bb.slice/crio-1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8 WatchSource:0}: Error finding container 1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8: Status 404 returned error can't find the container with id 1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8
Jul 25 00:44:50.146044 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:50.145950    2574 generic.go:356] "Generic (PLEG): container finished" podID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerID="20f9478b7a69cc95e007b69ee9ffea91247b44b186864bdc8ac9e1a236974186" exitCode=0
Jul 25 00:44:50.146179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:50.146026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm" event={"ID":"3a6cfbee-7f43-48d2-b2e7-7807068245bb","Type":"ContainerDied","Data":"20f9478b7a69cc95e007b69ee9ffea91247b44b186864bdc8ac9e1a236974186"}
Jul 25 00:44:50.146179 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:50.146092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm" event={"ID":"3a6cfbee-7f43-48d2-b2e7-7807068245bb","Type":"ContainerStarted","Data":"1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8"}
Jul 25 00:44:50.251494 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:50.251460    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:50.639303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:50.639270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:53.161617 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:53.161584    2574 generic.go:356] "Generic (PLEG): container finished" podID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerID="75794c28387e9dfb3d9ba56649e6c22a3ba836a97e7881ba426ff78276e8781b" exitCode=0
Jul 25 00:44:53.162065 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:53.161652    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm" event={"ID":"3a6cfbee-7f43-48d2-b2e7-7807068245bb","Type":"ContainerDied","Data":"75794c28387e9dfb3d9ba56649e6c22a3ba836a97e7881ba426ff78276e8781b"}
Jul 25 00:44:53.426150 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:53.426059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:53.481796 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:53.481761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:54.169039 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:54.169004    2574 generic.go:356] "Generic (PLEG): container finished" podID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerID="f7a397823de91978c9a980a0b583232a7f320a355e9e80cc35add7a59b9a1756" exitCode=0
Jul 25 00:44:54.169441 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:54.169084    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm" event={"ID":"3a6cfbee-7f43-48d2-b2e7-7807068245bb","Type":"ContainerDied","Data":"f7a397823de91978c9a980a0b583232a7f320a355e9e80cc35add7a59b9a1756"}
Jul 25 00:44:55.302237 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.302211    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:55.374693 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.374656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle\") pod \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") "
Jul 25 00:44:55.374871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.374727    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chtxp\" (UniqueName: \"kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp\") pod \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") "
Jul 25 00:44:55.374871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.374768    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util\") pod \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\" (UID: \"3a6cfbee-7f43-48d2-b2e7-7807068245bb\") "
Jul 25 00:44:55.375148 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.375118    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle" (OuterVolumeSpecName: "bundle") pod "3a6cfbee-7f43-48d2-b2e7-7807068245bb" (UID: "3a6cfbee-7f43-48d2-b2e7-7807068245bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:44:55.376916 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.376893    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp" (OuterVolumeSpecName: "kube-api-access-chtxp") pod "3a6cfbee-7f43-48d2-b2e7-7807068245bb" (UID: "3a6cfbee-7f43-48d2-b2e7-7807068245bb"). InnerVolumeSpecName "kube-api-access-chtxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:44:55.379344 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.379322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util" (OuterVolumeSpecName: "util") pod "3a6cfbee-7f43-48d2-b2e7-7807068245bb" (UID: "3a6cfbee-7f43-48d2-b2e7-7807068245bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:44:55.475975 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.475938    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:55.475975 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.475970    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a6cfbee-7f43-48d2-b2e7-7807068245bb-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:55.475975 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:55.475980    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chtxp\" (UniqueName: \"kubernetes.io/projected/3a6cfbee-7f43-48d2-b2e7-7807068245bb-kube-api-access-chtxp\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:44:56.179776 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:56.179741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm" event={"ID":"3a6cfbee-7f43-48d2-b2e7-7807068245bb","Type":"ContainerDied","Data":"1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8"}
Jul 25 00:44:56.179949 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:56.179790    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dbbf924c0e0fd9c67678bc8a9d3a958af1de9e69efa0932ecbc9fefb9128fc8"
Jul 25 00:44:56.179949 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:56.179754    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgvjm"
Jul 25 00:44:59.279806 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:59.279768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:59.844632 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:59.844597    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:44:59.944845 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:44:59.944812    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:01.216577 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216542    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"]
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216869    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="util"
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216880    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="util"
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216894    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="pull"
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216900    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="pull"
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216914    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="extract"
Jul 25 00:45:01.216969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216919    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="extract"
Jul 25 00:45:01.217151 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.216975    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6cfbee-7f43-48d2-b2e7-7807068245bb" containerName="extract"
Jul 25 00:45:01.220996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.220979    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.223235 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.223208    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 25 00:45:01.223235 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.223225    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8hkg9\""
Jul 25 00:45:01.224152 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.224131    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 25 00:45:01.227104 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.227081    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"]
Jul 25 00:45:01.327844 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.327801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sggr\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-kube-api-access-6sggr\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.328073 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.327853    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.429350 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.429315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sggr\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-kube-api-access-6sggr\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.429350 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.429350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.436866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.436837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sggr\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-kube-api-access-6sggr\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.436866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.436864    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7b5d277-923b-416d-9824-8f63622cee43-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rw6fm\" (UID: \"c7b5d277-923b-416d-9824-8f63622cee43\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.545670 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.545585    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"
Jul 25 00:45:01.680025 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:01.679997    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm"]
Jul 25 00:45:01.682570 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:01.682542    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7b5d277_923b_416d_9824_8f63622cee43.slice/crio-8ba2cbf1eb748d7b7afb52c150160e7f2ebac958f00b2ae33556d0b670c58a41 WatchSource:0}: Error finding container 8ba2cbf1eb748d7b7afb52c150160e7f2ebac958f00b2ae33556d0b670c58a41: Status 404 returned error can't find the container with id 8ba2cbf1eb748d7b7afb52c150160e7f2ebac958f00b2ae33556d0b670c58a41
Jul 25 00:45:02.208482 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:02.208442    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm" event={"ID":"c7b5d277-923b-416d-9824-8f63622cee43","Type":"ContainerStarted","Data":"8ba2cbf1eb748d7b7afb52c150160e7f2ebac958f00b2ae33556d0b670c58a41"}
Jul 25 00:45:06.231859 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:06.231803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm" event={"ID":"c7b5d277-923b-416d-9824-8f63622cee43","Type":"ContainerStarted","Data":"90462dc2e4aac5d4d5a1335fa4634533bdedc3eb1c159c02f981473c0a6110c5"}
Jul 25 00:45:06.248280 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:06.248231    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rw6fm" podStartSLOduration=0.918631875 podStartE2EDuration="5.248215518s" totalImagesPullingTime="4.329583643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:06.246171641 +0000 UTC m=+560.957590041" watchObservedRunningTime="2026-07-25 00:45:06.248215518 +0000 UTC m=+560.959633916"
Jul 25 00:45:06.785698 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:06.785658    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:06.864955 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:06.864913    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:07.568045 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:07.568008    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:07.667931 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:07.667899    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:09.154307 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.154273    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lgtmh"]
Jul 25 00:45:09.157719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.157699    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.159862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.159844    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-k27rl\""
Jul 25 00:45:09.167187 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.167163    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lgtmh"]
Jul 25 00:45:09.197987 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.197958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-bound-sa-token\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.198140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.197997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs6s9\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-kube-api-access-vs6s9\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.298996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.298959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-bound-sa-token\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.298996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.299001    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs6s9\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-kube-api-access-vs6s9\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.306672 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.306642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-bound-sa-token\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.306795 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.306735    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs6s9\" (UniqueName: \"kubernetes.io/projected/5d231237-c12d-4fc5-8a48-6a8111904872-kube-api-access-vs6s9\") pod \"cert-manager-55b4d954d5-lgtmh\" (UID: \"5d231237-c12d-4fc5-8a48-6a8111904872\") " pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.468175 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.468142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lgtmh"
Jul 25 00:45:09.594754 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.594728    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lgtmh"]
Jul 25 00:45:09.597148 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:09.597114    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d231237_c12d_4fc5_8a48_6a8111904872.slice/crio-cd92dd21286f780a929ff8974935f679387ba7f0a0427c3ce1bc8d0c27fa21ef WatchSource:0}: Error finding container cd92dd21286f780a929ff8974935f679387ba7f0a0427c3ce1bc8d0c27fa21ef: Status 404 returned error can't find the container with id cd92dd21286f780a929ff8974935f679387ba7f0a0427c3ce1bc8d0c27fa21ef
Jul 25 00:45:09.696719 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:09.696678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:10.250950 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.250910    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lgtmh" event={"ID":"5d231237-c12d-4fc5-8a48-6a8111904872","Type":"ContainerStarted","Data":"408d75fb50c49369f308280d80e5f65e2f371581deab05fdd01ccda868c9d279"}
Jul 25 00:45:10.250950 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.250956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lgtmh" event={"ID":"5d231237-c12d-4fc5-8a48-6a8111904872","Type":"ContainerStarted","Data":"cd92dd21286f780a929ff8974935f679387ba7f0a0427c3ce1bc8d0c27fa21ef"}
Jul 25 00:45:10.265363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.265315    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lgtmh" podStartSLOduration=1.265298024 podStartE2EDuration="1.265298024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:10.264110123 +0000 UTC m=+564.975528523" watchObservedRunningTime="2026-07-25 00:45:10.265298024 +0000 UTC m=+564.976716425"
Jul 25 00:45:10.495984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.495949    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"]
Jul 25 00:45:10.499628 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.499607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.501738 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.501667    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b6l4x\""
Jul 25 00:45:10.501863 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.501795    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 25 00:45:10.501935 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.501879    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 25 00:45:10.508031 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.508009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"]
Jul 25 00:45:10.613210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.613162    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.613410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.613217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4qx5\" (UniqueName: \"kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.613410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.613313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.714331 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.714293    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.714552 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.714388    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.714552 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.714421    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4qx5\" (UniqueName: \"kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.714807 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.714779    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.714862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.714780    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.721898 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.721873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4qx5\" (UniqueName: \"kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.810391 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.810303    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:10.942541 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:10.942448    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"]
Jul 25 00:45:10.944789 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:10.944761    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cdaa2a_6b01_40ee_854e_3a37e4a4dd97.slice/crio-c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291 WatchSource:0}: Error finding container c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291: Status 404 returned error can't find the container with id c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291
Jul 25 00:45:11.257045 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:11.257006    2574 generic.go:356] "Generic (PLEG): container finished" podID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerID="6e1a92fbe5c81f09e42f5d5c0ce68c500231c572cb85aaa9643cc6ddbcbc9556" exitCode=0
Jul 25 00:45:11.257444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:11.257095    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd" event={"ID":"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97","Type":"ContainerDied","Data":"6e1a92fbe5c81f09e42f5d5c0ce68c500231c572cb85aaa9643cc6ddbcbc9556"}
Jul 25 00:45:11.257444 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:11.257144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd" event={"ID":"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97","Type":"ContainerStarted","Data":"c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291"}
Jul 25 00:45:12.263932 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:12.263901    2574 generic.go:356] "Generic (PLEG): container finished" podID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerID="99ad22afeb39e1f9a2eeaa29ff17b7b54c4395e2d02206dbd2f50e5aee6e1a8b" exitCode=0
Jul 25 00:45:12.264328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:12.263989    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd" event={"ID":"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97","Type":"ContainerDied","Data":"99ad22afeb39e1f9a2eeaa29ff17b7b54c4395e2d02206dbd2f50e5aee6e1a8b"}
Jul 25 00:45:13.270750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:13.270717    2574 generic.go:356] "Generic (PLEG): container finished" podID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerID="1f8d0caca5906efdfd3fe3345bc896859293394b8e3fd04fb02007f5f0fa199f" exitCode=0
Jul 25 00:45:13.271121 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:13.270799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd" event={"ID":"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97","Type":"ContainerDied","Data":"1f8d0caca5906efdfd3fe3345bc896859293394b8e3fd04fb02007f5f0fa199f"}
Jul 25 00:45:14.410390 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.410359    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:14.549725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.549634    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4qx5\" (UniqueName: \"kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5\") pod \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") "
Jul 25 00:45:14.549725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.549725    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util\") pod \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") "
Jul 25 00:45:14.549910 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.549826    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle\") pod \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\" (UID: \"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97\") "
Jul 25 00:45:14.550542 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.550494    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle" (OuterVolumeSpecName: "bundle") pod "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" (UID: "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:45:14.551867 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.551848    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5" (OuterVolumeSpecName: "kube-api-access-j4qx5") pod "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" (UID: "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97"). InnerVolumeSpecName "kube-api-access-j4qx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:45:14.555160 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.555129    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util" (OuterVolumeSpecName: "util") pod "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" (UID: "85cdaa2a-6b01-40ee-854e-3a37e4a4dd97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:45:14.651110 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.651070    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:45:14.651110 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.651101    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:45:14.651110 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:14.651111    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4qx5\" (UniqueName: \"kubernetes.io/projected/85cdaa2a-6b01-40ee-854e-3a37e4a4dd97-kube-api-access-j4qx5\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:45:15.281979 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:15.281942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd" event={"ID":"85cdaa2a-6b01-40ee-854e-3a37e4a4dd97","Type":"ContainerDied","Data":"c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291"}
Jul 25 00:45:15.281979 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:15.281981    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcjnd"
Jul 25 00:45:15.282191 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:15.281995    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9ca8cd1a5476ed31b8b7cf84b61d63cc6988a8d623b4b863067766d38d42291"
Jul 25 00:45:21.654013 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:21.653973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:22.586990 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.586951    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"]
Jul 25 00:45:22.587303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587289    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="util"
Jul 25 00:45:22.587303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587303    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="util"
Jul 25 00:45:22.587407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587324    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="pull"
Jul 25 00:45:22.587407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587329    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="pull"
Jul 25 00:45:22.587407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587335    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="extract"
Jul 25 00:45:22.587407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587340    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="extract"
Jul 25 00:45:22.587572 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.587410    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85cdaa2a-6b01-40ee-854e-3a37e4a4dd97" containerName="extract"
Jul 25 00:45:22.593171 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.593154    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.595673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.595648    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 25 00:45:22.595948 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.595923    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 25 00:45:22.596057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.596007    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rcwf8\""
Jul 25 00:45:22.596057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.596018    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 25 00:45:22.596260 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.596245    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 25 00:45:22.607245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.607213    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"]
Jul 25 00:45:22.723670 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.723628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.724074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.723697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.724074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.723733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.724074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.723794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.724074 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.723850    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6c9\" (UniqueName: \"kubernetes.io/projected/341d8c67-6c80-4531-8786-5c321c007806-kube-api-access-cv6c9\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.824995 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.824953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.825169 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.825010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.825169 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.825053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.825169 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.825079    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6c9\" (UniqueName: \"kubernetes.io/projected/341d8c67-6c80-4531-8786-5c321c007806-kube-api-access-cv6c9\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.825280 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.825221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.825612 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.825586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-manifests\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.827594 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.827566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/341d8c67-6c80-4531-8786-5c321c007806-tmp\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.827703 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.827650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.827747 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.827723    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/341d8c67-6c80-4531-8786-5c321c007806-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.832727 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.832698    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6c9\" (UniqueName: \"kubernetes.io/projected/341d8c67-6c80-4531-8786-5c321c007806-kube-api-access-cv6c9\") pod \"opendatahub-operator-controller-manager-9b8579fc9-h8lxc\" (UID: \"341d8c67-6c80-4531-8786-5c321c007806\") " pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:22.903596 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:22.903493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:23.048727 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:23.048701    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"]
Jul 25 00:45:23.050736 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:23.050700    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341d8c67_6c80_4531_8786_5c321c007806.slice/crio-6e70aa3b6e91967f94ddda2104110ad440f6096c05dbfd16b1b9783912376296 WatchSource:0}: Error finding container 6e70aa3b6e91967f94ddda2104110ad440f6096c05dbfd16b1b9783912376296: Status 404 returned error can't find the container with id 6e70aa3b6e91967f94ddda2104110ad440f6096c05dbfd16b1b9783912376296
Jul 25 00:45:23.321906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:23.321868    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc" event={"ID":"341d8c67-6c80-4531-8786-5c321c007806","Type":"ContainerStarted","Data":"6e70aa3b6e91967f94ddda2104110ad440f6096c05dbfd16b1b9783912376296"}
Jul 25 00:45:23.836877 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:23.836838    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:26.340885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:26.340797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc" event={"ID":"341d8c67-6c80-4531-8786-5c321c007806","Type":"ContainerStarted","Data":"edb0e42bed18c564e55a131474c714abbb0f831b7605322d09a5e157da724b3e"}
Jul 25 00:45:27.348653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:27.348615    2574 generic.go:356] "Generic (PLEG): container finished" podID="341d8c67-6c80-4531-8786-5c321c007806" containerID="edb0e42bed18c564e55a131474c714abbb0f831b7605322d09a5e157da724b3e" exitCode=0
Jul 25 00:45:27.349215 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:27.349176    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc" event={"ID":"341d8c67-6c80-4531-8786-5c321c007806","Type":"ContainerDied","Data":"edb0e42bed18c564e55a131474c714abbb0f831b7605322d09a5e157da724b3e"}
Jul 25 00:45:28.357497 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:28.357459    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc" event={"ID":"341d8c67-6c80-4531-8786-5c321c007806","Type":"ContainerStarted","Data":"730b9c01c823ff93fbd12136de7f1b601c72368a10516b8040faa8ef5f06b419"}
Jul 25 00:45:28.357923 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:28.357630    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:28.381105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:28.381046    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc" podStartSLOduration=2.039534454 podStartE2EDuration="6.381026554s" totalImagesPullingTime="3.316147563s" totalInitContainerRuntime="1.025344537s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:28.379563747 +0000 UTC m=+583.090982146" watchObservedRunningTime="2026-07-25 00:45:28.381026554 +0000 UTC m=+583.092444953"
Jul 25 00:45:39.364393 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:39.364361    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9b8579fc9-h8lxc"
Jul 25 00:45:41.517760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:41.517717    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:41.713142 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:41.713104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:42.246099 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.246058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:42.310181 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.310141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:42.665751 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.665661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:42.731755 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.731718    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hxqch"]
Jul 25 00:45:42.740530 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.740481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:42.743972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.743944    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 25 00:45:42.744198 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.744175    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hxqch"]
Jul 25 00:45:42.746459 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.746432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:42.901656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.901623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfnfh\" (UniqueName: \"kubernetes.io/projected/3b8ecbeb-442e-4be4-af00-9137c82048ca-kube-api-access-rfnfh\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:42.901824 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:42.901743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ecbeb-442e-4be4-af00-9137c82048ca-tls-certs\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.002652 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.002614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ecbeb-442e-4be4-af00-9137c82048ca-tls-certs\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.002823 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.002676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfnfh\" (UniqueName: \"kubernetes.io/projected/3b8ecbeb-442e-4be4-af00-9137c82048ca-kube-api-access-rfnfh\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.005119 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.005093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ecbeb-442e-4be4-af00-9137c82048ca-tls-certs\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.009994 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.009970    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfnfh\" (UniqueName: \"kubernetes.io/projected/3b8ecbeb-442e-4be4-af00-9137c82048ca-kube-api-access-rfnfh\") pod \"model-serving-api-54d89bd5fc-hxqch\" (UID: \"3b8ecbeb-442e-4be4-af00-9137c82048ca\") " pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.020627 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.020591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:43.053978 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.053955    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:43.192746 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.192699    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hxqch"]
Jul 25 00:45:43.196226 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:43.196193    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8ecbeb_442e_4be4_af00_9137c82048ca.slice/crio-a31d19acbcf0c97e239629dbffedd2c7cfb5481b31af5f96c2d403c2002e1b11 WatchSource:0}: Error finding container a31d19acbcf0c97e239629dbffedd2c7cfb5481b31af5f96c2d403c2002e1b11: Status 404 returned error can't find the container with id a31d19acbcf0c97e239629dbffedd2c7cfb5481b31af5f96c2d403c2002e1b11
Jul 25 00:45:43.432244 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:43.432154    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch" event={"ID":"3b8ecbeb-442e-4be4-af00-9137c82048ca","Type":"ContainerStarted","Data":"a31d19acbcf0c97e239629dbffedd2c7cfb5481b31af5f96c2d403c2002e1b11"}
Jul 25 00:45:45.445771 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.445729    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch" event={"ID":"3b8ecbeb-442e-4be4-af00-9137c82048ca","Type":"ContainerStarted","Data":"7c6ba8127d45e50476d9893b753d1a6d1a7aabc2bcbdae6eb6b53f9bcc49ed2b"}
Jul 25 00:45:45.446161 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.445837    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:45.463869 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.463811    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch" podStartSLOduration=1.797425043 podStartE2EDuration="3.463791905s" totalImagesPullingTime="1.666366862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:45.459758632 +0000 UTC m=+600.171177032" watchObservedRunningTime="2026-07-25 00:45:45.463791905 +0000 UTC m=+600.175210305"
Jul 25 00:45:45.779122 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.779091    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:45:45.781191 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.781165    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:45:45.867421 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.867385    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:45.978822 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:45.978785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:47.386044 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.386008    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:47.406230 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.406199    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"]
Jul 25 00:45:47.409815 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.409786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.411904 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.411886    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8r4pz\""
Jul 25 00:45:47.412027 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.412004    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 25 00:45:47.412653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.412638    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 25 00:45:47.421086 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.421060    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"]
Jul 25 00:45:47.438924 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.438899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njshw\" (UniqueName: \"kubernetes.io/projected/84617057-0087-4e44-8a39-9d67dca933a7-kube-api-access-njshw\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.439041 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.438937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.439041 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.438958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84617057-0087-4e44-8a39-9d67dca933a7-tmp\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.540388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.540350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njshw\" (UniqueName: \"kubernetes.io/projected/84617057-0087-4e44-8a39-9d67dca933a7-kube-api-access-njshw\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.540388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.540399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.540693 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.540419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84617057-0087-4e44-8a39-9d67dca933a7-tmp\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.540693 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:47.540575    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 25 00:45:47.540693 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:47.540656    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs podName:84617057-0087-4e44-8a39-9d67dca933a7 nodeName:}" failed. No retries permitted until 2026-07-25 00:45:48.040631486 +0000 UTC m=+602.752049877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs") pod "kube-auth-proxy-68546f88c4-ddtrh" (UID: "84617057-0087-4e44-8a39-9d67dca933a7") : secret "kube-auth-proxy-tls" not found
Jul 25 00:45:47.542730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.542699    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84617057-0087-4e44-8a39-9d67dca933a7-tmp\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.548923 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.548894    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njshw\" (UniqueName: \"kubernetes.io/projected/84617057-0087-4e44-8a39-9d67dca933a7-kube-api-access-njshw\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:47.800216 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:47.800176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:48.046424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.046375    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:48.049054 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.049024    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84617057-0087-4e44-8a39-9d67dca933a7-tls-certs\") pod \"kube-auth-proxy-68546f88c4-ddtrh\" (UID: \"84617057-0087-4e44-8a39-9d67dca933a7\") " pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:48.299308 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.299277    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:45:48.303295 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.303270    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.304775 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.304751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:48.305534 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.305500    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 25 00:45:48.305626 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.305609    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ms2k8\""
Jul 25 00:45:48.305685 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.305669    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 25 00:45:48.305781 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.305763    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 25 00:45:48.305830 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.305796    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 25 00:45:48.313886 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.313861    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:45:48.320684 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.320665    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"
Jul 25 00:45:48.350065 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350220 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350068    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350220 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350115    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350220 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350171    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w472l\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.350551 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.350431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.404154 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.404124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w472l\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.451956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.453560 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.453184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.456231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.456182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.459125 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.459101    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.461772 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.459977    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh"]
Jul 25 00:45:48.461772 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.461754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w472l\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.463225 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.462656    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.464073 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.464048    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.464323 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.464301    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-788pr\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.465284 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:48.465255    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84617057_0087_4e44_8a39_9d67dca933a7.slice/crio-a93b9a77684c34909aa989b6221ac453673a2f9c7a3b23b83898493073ca7bdc WatchSource:0}: Error finding container a93b9a77684c34909aa989b6221ac453673a2f9c7a3b23b83898493073ca7bdc: Status 404 returned error can't find the container with id a93b9a77684c34909aa989b6221ac453673a2f9c7a3b23b83898493073ca7bdc
Jul 25 00:45:48.614635 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.614551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:45:48.746216 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:48.746188    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:45:48.747895 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:48.747864    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1279eff3_9fa9_4287_903d_9b0cedb98453.slice/crio-3181a54f93e58a772e2898ee5ecedcab9a15d4d265e78f5da71a8a4add6e253b WatchSource:0}: Error finding container 3181a54f93e58a772e2898ee5ecedcab9a15d4d265e78f5da71a8a4add6e253b: Status 404 returned error can't find the container with id 3181a54f93e58a772e2898ee5ecedcab9a15d4d265e78f5da71a8a4add6e253b
Jul 25 00:45:49.477822 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:49.477774    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh" event={"ID":"84617057-0087-4e44-8a39-9d67dca933a7","Type":"ContainerStarted","Data":"a93b9a77684c34909aa989b6221ac453673a2f9c7a3b23b83898493073ca7bdc"}
Jul 25 00:45:49.480091 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:49.480054    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" event={"ID":"1279eff3-9fa9-4287-903d-9b0cedb98453","Type":"ContainerStarted","Data":"3181a54f93e58a772e2898ee5ecedcab9a15d4d265e78f5da71a8a4add6e253b"}
Jul 25 00:45:49.590085 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:49.590052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:51.494072 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:51.494026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh" event={"ID":"84617057-0087-4e44-8a39-9d67dca933a7","Type":"ContainerStarted","Data":"0bb9e89fa0c10443b832d0232d64ffefd76ecd1d45a15f71bcf056cf9c296256"}
Jul 25 00:45:51.514354 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:51.514294    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68546f88c4-ddtrh" podStartSLOduration=1.64743912 podStartE2EDuration="4.514268512s" totalImagesPullingTime="2.866829392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:51.512119917 +0000 UTC m=+606.223538319" watchObservedRunningTime="2026-07-25 00:45:51.514268512 +0000 UTC m=+606.225686910"
Jul 25 00:45:52.164406 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.164367    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:52.219250 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.219217    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"]
Jul 25 00:45:52.223544 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.223521    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.225847 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.225813    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6q756\""
Jul 25 00:45:52.225972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.225907    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 25 00:45:52.233189 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.233161    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"]
Jul 25 00:45:52.239776 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.239754    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"]
Jul 25 00:45:52.244192 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.244170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.247134 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.247116    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 25 00:45:52.247346 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.247314    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-45b2t\""
Jul 25 00:45:52.261226 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.261200    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"]
Jul 25 00:45:52.290052 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.290019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.290303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.290271    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh6rj\" (UniqueName: \"kubernetes.io/projected/d4349849-c613-4672-8d93-e32b8b610203-kube-api-access-nh6rj\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.290428 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.290407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.290531 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.290459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4bsc\" (UniqueName: \"kubernetes.io/projected/5744f514-08af-403d-b8bb-f4d009d5971b-kube-api-access-x4bsc\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.300702 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.300491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:52.391480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.391444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.391480 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.391485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4bsc\" (UniqueName: \"kubernetes.io/projected/5744f514-08af-403d-b8bb-f4d009d5971b-kube-api-access-x4bsc\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.391741 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.391564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.391741 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:52.391628    2574 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 25 00:45:52.391741 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.391612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh6rj\" (UniqueName: \"kubernetes.io/projected/d4349849-c613-4672-8d93-e32b8b610203-kube-api-access-nh6rj\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.391741 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:52.391696    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert podName:5744f514-08af-403d-b8bb-f4d009d5971b nodeName:}" failed. No retries permitted until 2026-07-25 00:45:52.891676313 +0000 UTC m=+607.603094692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert") pod "llmisvc-controller-manager-777c6bd7d5-l4l7j" (UID: "5744f514-08af-403d-b8bb-f4d009d5971b") : secret "llmisvc-webhook-server-cert" not found
Jul 25 00:45:52.391741 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:52.391713    2574 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 25 00:45:52.392031 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:45:52.391763    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert podName:d4349849-c613-4672-8d93-e32b8b610203 nodeName:}" failed. No retries permitted until 2026-07-25 00:45:52.891748055 +0000 UTC m=+607.603166453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert") pod "kserve-controller-manager-d6f4b65b8-tsbnn" (UID: "d4349849-c613-4672-8d93-e32b8b610203") : secret "kserve-webhook-server-cert" not found
Jul 25 00:45:52.401090 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.401047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4bsc\" (UniqueName: \"kubernetes.io/projected/5744f514-08af-403d-b8bb-f4d009d5971b-kube-api-access-x4bsc\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.401246 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.401145    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh6rj\" (UniqueName: \"kubernetes.io/projected/d4349849-c613-4672-8d93-e32b8b610203-kube-api-access-nh6rj\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.563219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.563172    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:52.754556 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.754518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:52.897374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.897270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:52.897374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.897360    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.899966 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.899934    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5744f514-08af-403d-b8bb-f4d009d5971b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l4l7j\" (UID: \"5744f514-08af-403d-b8bb-f4d009d5971b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:52.900072 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:52.899938    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4349849-c613-4672-8d93-e32b8b610203-cert\") pod \"kserve-controller-manager-d6f4b65b8-tsbnn\" (UID: \"d4349849-c613-4672-8d93-e32b8b610203\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:53.137703 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.137666    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:53.155661 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.155621    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:53.290583 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.290490    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"]
Jul 25 00:45:53.296368 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:53.296335    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4349849_c613_4672_8d93_e32b8b610203.slice/crio-85a3028217ceb3dc296043d9eceec0b6372a6c931750a8d760700e949f25383a WatchSource:0}: Error finding container 85a3028217ceb3dc296043d9eceec0b6372a6c931750a8d760700e949f25383a: Status 404 returned error can't find the container with id 85a3028217ceb3dc296043d9eceec0b6372a6c931750a8d760700e949f25383a
Jul 25 00:45:53.320969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.314187    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"]
Jul 25 00:45:53.325217 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:53.325187    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5744f514_08af_403d_b8bb_f4d009d5971b.slice/crio-5733f3fca06c85196870dd2f29d07a63ddd1b8a7dea35f3d949fc6f66a66545d WatchSource:0}: Error finding container 5733f3fca06c85196870dd2f29d07a63ddd1b8a7dea35f3d949fc6f66a66545d: Status 404 returned error can't find the container with id 5733f3fca06c85196870dd2f29d07a63ddd1b8a7dea35f3d949fc6f66a66545d
Jul 25 00:45:53.435925 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.435837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:53.506917 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.506881    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn" event={"ID":"d4349849-c613-4672-8d93-e32b8b610203","Type":"ContainerStarted","Data":"85a3028217ceb3dc296043d9eceec0b6372a6c931750a8d760700e949f25383a"}
Jul 25 00:45:53.507959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.507932    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j" event={"ID":"5744f514-08af-403d-b8bb-f4d009d5971b","Type":"ContainerStarted","Data":"5733f3fca06c85196870dd2f29d07a63ddd1b8a7dea35f3d949fc6f66a66545d"}
Jul 25 00:45:53.536582 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.536552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:53.688412 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:53.688325    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:56.457149 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.457115    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hxqch"
Jul 25 00:45:56.530111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.530073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn" event={"ID":"d4349849-c613-4672-8d93-e32b8b610203","Type":"ContainerStarted","Data":"9ce5648b53ee40bb75fb1ed4525888b74ac6f52578a96687dfbbc6c05ce300d1"}
Jul 25 00:45:56.530278 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.530249    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:45:56.561760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.561703    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn" podStartSLOduration=2.073158951 podStartE2EDuration="4.561681487s" totalImagesPullingTime="2.488522536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:56.559196528 +0000 UTC m=+611.270614928" watchObservedRunningTime="2026-07-25 00:45:56.561681487 +0000 UTC m=+611.273099887"
Jul 25 00:45:56.768619 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.768526    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:56.862262 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:56.862229    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:57.433363 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.433328    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"]
Jul 25 00:45:57.436947 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.436920    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.439328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.439297    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-44tj5\""
Jul 25 00:45:57.449262 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.449239    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"]
Jul 25 00:45:57.463076 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.463047    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:57.542956 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.542825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.542956 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.542868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543222 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8h9r\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-kube-api-access-r8h9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543222 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543222 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543222 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543438 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543438 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.543438 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.543353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.548157 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.548079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j" event={"ID":"5744f514-08af-403d-b8bb-f4d009d5971b","Type":"ContainerStarted","Data":"ff1a2bddcd79d1155f267bc6762fc2c71412f94932352a856d5a3b978cf2a89d"}
Jul 25 00:45:57.548361 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.548334    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:45:57.563434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.563382    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j" podStartSLOduration=2.356665339 podStartE2EDuration="5.56336237s" totalImagesPullingTime="3.206697031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:45:57.562414702 +0000 UTC m=+612.273833101" watchObservedRunningTime="2026-07-25 00:45:57.56336237 +0000 UTC m=+612.274780772"
Jul 25 00:45:57.644930 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.644886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645147 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.644952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645147 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.644982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645147 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645147 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645147 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645308    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8h9r\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-kube-api-access-r8h9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645410 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645595 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645409    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.645595 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.645483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.646093 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.646071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.646213 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.646071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.647633 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.647610    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.647914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.647896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.654234 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.654216    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.654596 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.654573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8h9r\" (UniqueName: \"kubernetes.io/projected/d36a9860-5bec-4dc4-8da0-a8bff55d3ba7-kube-api-access-r8h9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5\" (UID: \"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.663985 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.663959    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:57.749292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.749254    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:45:57.763564 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.763495    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:45:57.895437 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.895384    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"]
Jul 25 00:45:57.897909 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:45:57.897884    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36a9860_5bec_4dc4_8da0_a8bff55d3ba7.slice/crio-6ee811f068ca80599a799c01e6d651a110327e7e727a30814e329279977d01c9 WatchSource:0}: Error finding container 6ee811f068ca80599a799c01e6d651a110327e7e727a30814e329279977d01c9: Status 404 returned error can't find the container with id 6ee811f068ca80599a799c01e6d651a110327e7e727a30814e329279977d01c9
Jul 25 00:45:57.900328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:57.900313    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:45:58.554356 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:45:58.554317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5" event={"ID":"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7","Type":"ContainerStarted","Data":"6ee811f068ca80599a799c01e6d651a110327e7e727a30814e329279977d01c9"}
Jul 25 00:46:00.094076 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:00.094035    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:46:01.515555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:01.515501    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 00:46:01.515841 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:01.515592    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 00:46:01.515841 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:01.515623    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 00:46:01.779463 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:01.779377    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:01.878969 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:01.878930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:02.231541 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:02.231466    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:02.582249 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:02.582163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5" event={"ID":"d36a9860-5bec-4dc4-8da0-a8bff55d3ba7","Type":"ContainerStarted","Data":"38e8385853d69789f941783c8d048504247d0201a22cd9323e987f004aac24e4"}
Jul 25 00:46:02.601256 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:02.601201    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5" podStartSLOduration=1.986403491 podStartE2EDuration="5.601185993s" totalImagesPullingTime="3.614782502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:46:02.600190652 +0000 UTC m=+617.311609079" watchObservedRunningTime="2026-07-25 00:46:02.601185993 +0000 UTC m=+617.312604391"
Jul 25 00:46:02.750185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:02.750143    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:46:02.755432 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:02.755399    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:46:03.587525 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:03.587484    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:46:03.588396 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:03.588377    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5"
Jul 25 00:46:16.861037 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:16.860999    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 00:46:16.861312 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:16.861070    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 00:46:17.665647 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.665607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" event={"ID":"1279eff3-9fa9-4287-903d-9b0cedb98453","Type":"ContainerStarted","Data":"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"}
Jul 25 00:46:17.665862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.665653    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" podUID="1279eff3-9fa9-4287-903d-9b0cedb98453" containerName="discovery" containerID="cri-o://85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a" gracePeriod=30
Jul 25 00:46:17.665862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.665783    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:46:17.667397 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.667376    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:46:17.684667 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.684625    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" podStartSLOduration=1.573882 podStartE2EDuration="29.684610767s" totalImagesPullingTime="28.110728767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:46:17.684157887 +0000 UTC m=+632.395576297" watchObservedRunningTime="2026-07-25 00:46:17.684610767 +0000 UTC m=+632.396029167"
Jul 25 00:46:17.923259 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:17.923194    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:46:18.044079 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044290 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044092    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044290 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044290 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044211    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w472l\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044290 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044234    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044491 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044295    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044491 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044321    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token\") pod \"1279eff3-9fa9-4287-903d-9b0cedb98453\" (UID: \"1279eff3-9fa9-4287-903d-9b0cedb98453\") "
Jul 25 00:46:18.044739 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.044703    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:46:18.046788 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token" (OuterVolumeSpecName: "istio-token") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:46:18.046788 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l" (OuterVolumeSpecName: "kube-api-access-w472l") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "kube-api-access-w472l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:46:18.046972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046894    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:46:18.046972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046894    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs" (OuterVolumeSpecName: "local-certs") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:46:18.046972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046928    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts" (OuterVolumeSpecName: "cacerts") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:46:18.046972 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.046964    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1279eff3-9fa9-4287-903d-9b0cedb98453" (UID: "1279eff3-9fa9-4287-903d-9b0cedb98453"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:46:18.145927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145893    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-kubeconfig\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.145927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145923    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-cacerts\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.145927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145932    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-dns-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.146172 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145942    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w472l\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-kube-api-access-w472l\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.146172 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145954    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1279eff3-9fa9-4287-903d-9b0cedb98453-local-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.146172 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145963    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-csr-ca-configmap\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.146172 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.145972    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1279eff3-9fa9-4287-903d-9b0cedb98453-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:18.672336 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.672301    2574 generic.go:356] "Generic (PLEG): container finished" podID="1279eff3-9fa9-4287-903d-9b0cedb98453" containerID="85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a" exitCode=0
Jul 25 00:46:18.672533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.672352    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"
Jul 25 00:46:18.672533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.672376    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" event={"ID":"1279eff3-9fa9-4287-903d-9b0cedb98453","Type":"ContainerDied","Data":"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"}
Jul 25 00:46:18.672533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.672464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr" event={"ID":"1279eff3-9fa9-4287-903d-9b0cedb98453","Type":"ContainerDied","Data":"3181a54f93e58a772e2898ee5ecedcab9a15d4d265e78f5da71a8a4add6e253b"}
Jul 25 00:46:18.672533 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.672487    2574 scope.go:117] "RemoveContainer" containerID="85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"
Jul 25 00:46:18.682653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.682634    2574 scope.go:117] "RemoveContainer" containerID="85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"
Jul 25 00:46:18.682960 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:46:18.682940    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a\": container with ID starting with 85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a not found: ID does not exist" containerID="85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"
Jul 25 00:46:18.683012 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.682970    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a"} err="failed to get container status \"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a\": rpc error: code = NotFound desc = could not find container \"85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a\": container with ID starting with 85e656c95af657aef0fcf46268b726bf11237347073ad8ebf2c5b54a90526b0a not found: ID does not exist"
Jul 25 00:46:18.694759 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.694687    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:46:18.698314 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:18.698282    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-788pr"]
Jul 25 00:46:19.932962 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:19.932931    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1279eff3-9fa9-4287-903d-9b0cedb98453" path="/var/lib/kubelet/pods/1279eff3-9fa9-4287-903d-9b0cedb98453/volumes"
Jul 25 00:46:27.553422 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:27.553388    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tsbnn"
Jul 25 00:46:28.557399 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:28.557368    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l4l7j"
Jul 25 00:46:43.062877 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.062831    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:43.153673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.153640    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:43.184729 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.184699    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:43.185193 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.185170    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1279eff3-9fa9-4287-903d-9b0cedb98453" containerName="discovery"
Jul 25 00:46:43.185334 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.185206    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1279eff3-9fa9-4287-903d-9b0cedb98453" containerName="discovery"
Jul 25 00:46:43.185397 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.185366    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1279eff3-9fa9-4287-903d-9b0cedb98453" containerName="discovery"
Jul 25 00:46:43.188409 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.188382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:43.190773 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.190751    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 25 00:46:43.190925 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.190880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9jd7t\""
Jul 25 00:46:43.190996 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.190936    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 25 00:46:43.197253 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.197100    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:43.225324 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.225276    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:43.262965 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.262931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54x2\" (UniqueName: \"kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2\") pod \"kuadrant-operator-catalog-tdt5x\" (UID: \"3a51bb72-7200-4821-a57c-6a0188d0b982\") " pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:43.363732 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.363659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54x2\" (UniqueName: \"kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2\") pod \"kuadrant-operator-catalog-tdt5x\" (UID: \"3a51bb72-7200-4821-a57c-6a0188d0b982\") " pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:43.375687 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.375658    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54x2\" (UniqueName: \"kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2\") pod \"kuadrant-operator-catalog-tdt5x\" (UID: \"3a51bb72-7200-4821-a57c-6a0188d0b982\") " pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:43.500565 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.500502    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:43.551212 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.551172    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:43.635345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.635318    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:43.638202 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:46:43.638175    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a51bb72_7200_4821_a57c_6a0188d0b982.slice/crio-5e54eb4c4176d25eae175080a1ebdbcb0cea142b46febc0b0c021b20a44664f5 WatchSource:0}: Error finding container 5e54eb4c4176d25eae175080a1ebdbcb0cea142b46febc0b0c021b20a44664f5: Status 404 returned error can't find the container with id 5e54eb4c4176d25eae175080a1ebdbcb0cea142b46febc0b0c021b20a44664f5
Jul 25 00:46:43.760031 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.760002    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7cd8"]
Jul 25 00:46:43.764545 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.764527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:43.770629 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.770604    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7cd8"]
Jul 25 00:46:43.804882 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.804855    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" event={"ID":"3a51bb72-7200-4821-a57c-6a0188d0b982","Type":"ContainerStarted","Data":"5e54eb4c4176d25eae175080a1ebdbcb0cea142b46febc0b0c021b20a44664f5"}
Jul 25 00:46:43.868720 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.868691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96txd\" (UniqueName: \"kubernetes.io/projected/a91d193f-4283-44b9-97df-2d50bf9e28d8-kube-api-access-96txd\") pod \"kuadrant-operator-catalog-w7cd8\" (UID: \"a91d193f-4283-44b9-97df-2d50bf9e28d8\") " pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:43.969692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.969664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96txd\" (UniqueName: \"kubernetes.io/projected/a91d193f-4283-44b9-97df-2d50bf9e28d8-kube-api-access-96txd\") pod \"kuadrant-operator-catalog-w7cd8\" (UID: \"a91d193f-4283-44b9-97df-2d50bf9e28d8\") " pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:43.976888 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:43.976869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96txd\" (UniqueName: \"kubernetes.io/projected/a91d193f-4283-44b9-97df-2d50bf9e28d8-kube-api-access-96txd\") pod \"kuadrant-operator-catalog-w7cd8\" (UID: \"a91d193f-4283-44b9-97df-2d50bf9e28d8\") " pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:44.075304 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:44.075272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:44.216397 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:44.216372    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w7cd8"]
Jul 25 00:46:44.229253 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:46:44.229178    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91d193f_4283_44b9_97df_2d50bf9e28d8.slice/crio-e14dd1e5e34ca86367fe15a069d098419c57c1e246e4aeed1139026b26e8307f WatchSource:0}: Error finding container e14dd1e5e34ca86367fe15a069d098419c57c1e246e4aeed1139026b26e8307f: Status 404 returned error can't find the container with id e14dd1e5e34ca86367fe15a069d098419c57c1e246e4aeed1139026b26e8307f
Jul 25 00:46:44.813529 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:44.813478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8" event={"ID":"a91d193f-4283-44b9-97df-2d50bf9e28d8","Type":"ContainerStarted","Data":"e14dd1e5e34ca86367fe15a069d098419c57c1e246e4aeed1139026b26e8307f"}
Jul 25 00:46:45.820185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:45.820131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" event={"ID":"3a51bb72-7200-4821-a57c-6a0188d0b982","Type":"ContainerStarted","Data":"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"}
Jul 25 00:46:45.820185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:45.820158    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" podUID="3a51bb72-7200-4821-a57c-6a0188d0b982" containerName="registry-server" containerID="cri-o://53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf" gracePeriod=2
Jul 25 00:46:45.821951 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:45.821922    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8" event={"ID":"a91d193f-4283-44b9-97df-2d50bf9e28d8","Type":"ContainerStarted","Data":"a36a9d257ee2f3d72ddca3af97da04a4d639194631f76a7bb34dc019a24081f7"}
Jul 25 00:46:45.835987 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:45.835931    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" podStartSLOduration=0.953711846 podStartE2EDuration="2.83591874s" totalImagesPullingTime="1.882206894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:46:45.834126381 +0000 UTC m=+660.545544790" watchObservedRunningTime="2026-07-25 00:46:45.83591874 +0000 UTC m=+660.547337138"
Jul 25 00:46:46.069062 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.069037    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:46.085124 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.085058    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8" podStartSLOduration=1.79323767 podStartE2EDuration="3.085040263s" totalImagesPullingTime="1.291802593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:46:45.847793411 +0000 UTC m=+660.559211822" watchObservedRunningTime="2026-07-25 00:46:46.085040263 +0000 UTC m=+660.796458661"
Jul 25 00:46:46.198575 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.198530    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b54x2\" (UniqueName: \"kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2\") pod \"3a51bb72-7200-4821-a57c-6a0188d0b982\" (UID: \"3a51bb72-7200-4821-a57c-6a0188d0b982\") "
Jul 25 00:46:46.200754 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.200728    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2" (OuterVolumeSpecName: "kube-api-access-b54x2") pod "3a51bb72-7200-4821-a57c-6a0188d0b982" (UID: "3a51bb72-7200-4821-a57c-6a0188d0b982"). InnerVolumeSpecName "kube-api-access-b54x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:46:46.299914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.299833    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b54x2\" (UniqueName: \"kubernetes.io/projected/3a51bb72-7200-4821-a57c-6a0188d0b982-kube-api-access-b54x2\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:46:46.828010 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.827969    2574 generic.go:356] "Generic (PLEG): container finished" podID="3a51bb72-7200-4821-a57c-6a0188d0b982" containerID="53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf" exitCode=0
Jul 25 00:46:46.828424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.828038    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x"
Jul 25 00:46:46.828424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.828055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" event={"ID":"3a51bb72-7200-4821-a57c-6a0188d0b982","Type":"ContainerDied","Data":"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"}
Jul 25 00:46:46.828424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.828135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tdt5x" event={"ID":"3a51bb72-7200-4821-a57c-6a0188d0b982","Type":"ContainerDied","Data":"5e54eb4c4176d25eae175080a1ebdbcb0cea142b46febc0b0c021b20a44664f5"}
Jul 25 00:46:46.828424 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.828157    2574 scope.go:117] "RemoveContainer" containerID="53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"
Jul 25 00:46:46.837856 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.837835    2574 scope.go:117] "RemoveContainer" containerID="53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"
Jul 25 00:46:46.838099 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:46:46.838083    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf\": container with ID starting with 53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf not found: ID does not exist" containerID="53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"
Jul 25 00:46:46.838161 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.838105    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf"} err="failed to get container status \"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf\": rpc error: code = NotFound desc = could not find container \"53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf\": container with ID starting with 53af2324a6fa73c0d7c3d38cd03f79af6fb8b37dbd702b9dbb0544fd71e52fdf not found: ID does not exist"
Jul 25 00:46:46.848911 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.848882    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:46.850799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.850780    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tdt5x"]
Jul 25 00:46:46.957330 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:46.957284    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:47.004279 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:47.004247    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:47.932744 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:47.932710    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a51bb72-7200-4821-a57c-6a0188d0b982" path="/var/lib/kubelet/pods/3a51bb72-7200-4821-a57c-6a0188d0b982/volumes"
Jul 25 00:46:48.913081 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:48.913044    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:48.959014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:48.958983    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:46:54.075702 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:54.075651    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:54.076157 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:54.075728    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:54.097254 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:54.097227    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:54.891297 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:54.891272    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-w7cd8"
Jul 25 00:46:58.550517 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.550465    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"]
Jul 25 00:46:58.551061 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.551042    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a51bb72-7200-4821-a57c-6a0188d0b982" containerName="registry-server"
Jul 25 00:46:58.551061 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.551062    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a51bb72-7200-4821-a57c-6a0188d0b982" containerName="registry-server"
Jul 25 00:46:58.551191 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.551140    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a51bb72-7200-4821-a57c-6a0188d0b982" containerName="registry-server"
Jul 25 00:46:58.557545 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.557478    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.559685 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.559660    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9qz72\""
Jul 25 00:46:58.560330 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.560305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"]
Jul 25 00:46:58.715270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.715228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m49q\" (UniqueName: \"kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.715551 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.715368    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.715551 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.715429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.816165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.816068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.816165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.816130    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.816356 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.816305    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m49q\" (UniqueName: \"kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.816467 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.816452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.816531 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.816495    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.824089 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.824065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m49q\" (UniqueName: \"kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.868777 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.868754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:46:58.998613 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:58.998572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"]
Jul 25 00:46:59.000342 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:46:59.000320    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4375d1c_f127_45b7_961c_96c979d96cb5.slice/crio-2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339 WatchSource:0}: Error finding container 2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339: Status 404 returned error can't find the container with id 2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339
Jul 25 00:46:59.152177 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.152146    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"]
Jul 25 00:46:59.155782 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.155760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.163083 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.163060    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"]
Jul 25 00:46:59.321698 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.321665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.321871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.321724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.321871 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.321779    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txjl\" (UniqueName: \"kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.423206 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.423104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.423206 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.423153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txjl\" (UniqueName: \"kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.423445 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.423295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.423541 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.423476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.423673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.423654    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.431637 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.431619    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txjl\" (UniqueName: \"kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.474976 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.474952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:46:59.551174 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.551144    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"]
Jul 25 00:46:59.556746 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.556725    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.563488 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.563461    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"]
Jul 25 00:46:59.631318 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.631291    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"]
Jul 25 00:46:59.633011 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:46:59.632982    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda62d501_2e21_41c7_9d61_82a088ac7f46.slice/crio-edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1 WatchSource:0}: Error finding container edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1: Status 404 returned error can't find the container with id edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1
Jul 25 00:46:59.725751 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.725723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.725902 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.725784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtff\" (UniqueName: \"kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.725902 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.725820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.827146 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.827119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtff\" (UniqueName: \"kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.827297 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.827156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.827297 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.827243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.827616 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.827595    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.827707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.827685    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.835213 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.835178    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtff\" (UniqueName: \"kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.874048 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.874027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:46:59.898555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.898527    2574 generic.go:356] "Generic (PLEG): container finished" podID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerID="41e794c260d027b135c5eedf9949faa246771dab79f9a1f07c0660fe0031cecf" exitCode=0
Jul 25 00:46:59.898666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.898609    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6" event={"ID":"e4375d1c-f127-45b7-961c-96c979d96cb5","Type":"ContainerDied","Data":"41e794c260d027b135c5eedf9949faa246771dab79f9a1f07c0660fe0031cecf"}
Jul 25 00:46:59.898666 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.898662    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6" event={"ID":"e4375d1c-f127-45b7-961c-96c979d96cb5","Type":"ContainerStarted","Data":"2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339"}
Jul 25 00:46:59.900299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.900276    2574 generic.go:356] "Generic (PLEG): container finished" podID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerID="531809bfab337529c21cfe32c71c7da8da206bddb3db8f0e7bee32bffb65e48f" exitCode=0
Jul 25 00:46:59.900386 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.900365    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm" event={"ID":"da62d501-2e21-41c7-9d61-82a088ac7f46","Type":"ContainerDied","Data":"531809bfab337529c21cfe32c71c7da8da206bddb3db8f0e7bee32bffb65e48f"}
Jul 25 00:46:59.900423 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.900397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm" event={"ID":"da62d501-2e21-41c7-9d61-82a088ac7f46","Type":"ContainerStarted","Data":"edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1"}
Jul 25 00:46:59.958466 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.958437    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"]
Jul 25 00:46:59.963300 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.963282    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:46:59.972903 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:46:59.972876    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"]
Jul 25 00:47:00.021218 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.021175    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"]
Jul 25 00:47:00.022919 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:00.022889    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae843d58_08aa_4c34_9720_ddea4050b699.slice/crio-627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35 WatchSource:0}: Error finding container 627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35: Status 404 returned error can't find the container with id 627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35
Jul 25 00:47:00.130471 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.130446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzc4\" (UniqueName: \"kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.130610 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.130521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.130656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.130623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.232214 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.232117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.232214 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.232194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzc4\" (UniqueName: \"kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.232456 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.232229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.232574 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.232551    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.232645 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.232586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.239917 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.239899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzc4\" (UniqueName: \"kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.277913 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.277881    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:00.417853 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.417818    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"]
Jul 25 00:47:00.418988 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:00.418962    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fa2ac6_fdf6_4c8d_8507_9011420a4c73.slice/crio-1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0 WatchSource:0}: Error finding container 1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0: Status 404 returned error can't find the container with id 1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0
Jul 25 00:47:00.907747 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.907715    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae843d58-08aa-4c34-9720-ddea4050b699" containerID="ba905dedcab1ef0e2f70535d06f47e9d566b55fd58f1708c61f31fb25c0abc57" exitCode=0
Jul 25 00:47:00.908135 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.907791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z" event={"ID":"ae843d58-08aa-4c34-9720-ddea4050b699","Type":"ContainerDied","Data":"ba905dedcab1ef0e2f70535d06f47e9d566b55fd58f1708c61f31fb25c0abc57"}
Jul 25 00:47:00.908135 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.907842    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z" event={"ID":"ae843d58-08aa-4c34-9720-ddea4050b699","Type":"ContainerStarted","Data":"627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35"}
Jul 25 00:47:00.909550 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.909500    2574 generic.go:356] "Generic (PLEG): container finished" podID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerID="cf1ab570690a045b5386854c772dc56cb2c379a9e7ecfcc7a4c068ae6dfba3a2" exitCode=0
Jul 25 00:47:00.909550 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.909543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd" event={"ID":"56fa2ac6-fdf6-4c8d-8507-9011420a4c73","Type":"ContainerDied","Data":"cf1ab570690a045b5386854c772dc56cb2c379a9e7ecfcc7a4c068ae6dfba3a2"}
Jul 25 00:47:00.909716 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:00.909585    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd" event={"ID":"56fa2ac6-fdf6-4c8d-8507-9011420a4c73","Type":"ContainerStarted","Data":"1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0"}
Jul 25 00:47:01.916941 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.916853    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae843d58-08aa-4c34-9720-ddea4050b699" containerID="4dba05afe96e7e1ecd47793b40a3877dc546b35c41add783266995f9c9c3dcfd" exitCode=0
Jul 25 00:47:01.917327 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.916939    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z" event={"ID":"ae843d58-08aa-4c34-9720-ddea4050b699","Type":"ContainerDied","Data":"4dba05afe96e7e1ecd47793b40a3877dc546b35c41add783266995f9c9c3dcfd"}
Jul 25 00:47:01.918861 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.918840    2574 generic.go:356] "Generic (PLEG): container finished" podID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerID="053aa72aa27fc57e976996b4a3b6ac9899ea95a4b2da14356e018e803216b406" exitCode=0
Jul 25 00:47:01.918977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.918926    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6" event={"ID":"e4375d1c-f127-45b7-961c-96c979d96cb5","Type":"ContainerDied","Data":"053aa72aa27fc57e976996b4a3b6ac9899ea95a4b2da14356e018e803216b406"}
Jul 25 00:47:01.920929 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.920910    2574 generic.go:356] "Generic (PLEG): container finished" podID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerID="d80bc95d2ace669b38c26d9b3ae2cc8024e9ed37923955df99bb6b9b99feb82a" exitCode=0
Jul 25 00:47:01.921037 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.920990    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm" event={"ID":"da62d501-2e21-41c7-9d61-82a088ac7f46","Type":"ContainerDied","Data":"d80bc95d2ace669b38c26d9b3ae2cc8024e9ed37923955df99bb6b9b99feb82a"}
Jul 25 00:47:01.923088 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.923067    2574 generic.go:356] "Generic (PLEG): container finished" podID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerID="0564abfbdded0d638f7e225450a77032427bba04ed3a5f2e541fc05e332782f8" exitCode=0
Jul 25 00:47:01.923169 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:01.923149    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd" event={"ID":"56fa2ac6-fdf6-4c8d-8507-9011420a4c73","Type":"ContainerDied","Data":"0564abfbdded0d638f7e225450a77032427bba04ed3a5f2e541fc05e332782f8"}
Jul 25 00:47:02.930750 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.930719    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae843d58-08aa-4c34-9720-ddea4050b699" containerID="e2d037e526bf62e45fdec859620c2f19be7eccb0b82d982bf78da9740437fe41" exitCode=0
Jul 25 00:47:02.931128 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.930799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z" event={"ID":"ae843d58-08aa-4c34-9720-ddea4050b699","Type":"ContainerDied","Data":"e2d037e526bf62e45fdec859620c2f19be7eccb0b82d982bf78da9740437fe41"}
Jul 25 00:47:02.932758 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.932733    2574 generic.go:356] "Generic (PLEG): container finished" podID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerID="f00c4c906f4cfa4e8d085ad46736f9c55e91579d4a5bee4df99adb93f912fc65" exitCode=0
Jul 25 00:47:02.932866 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.932805    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6" event={"ID":"e4375d1c-f127-45b7-961c-96c979d96cb5","Type":"ContainerDied","Data":"f00c4c906f4cfa4e8d085ad46736f9c55e91579d4a5bee4df99adb93f912fc65"}
Jul 25 00:47:02.934928 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.934910    2574 generic.go:356] "Generic (PLEG): container finished" podID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerID="889bbfaffe0303e644ea28613cc80e4872119287744a2cd1587206a97a420c41" exitCode=0
Jul 25 00:47:02.935012 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.934988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm" event={"ID":"da62d501-2e21-41c7-9d61-82a088ac7f46","Type":"ContainerDied","Data":"889bbfaffe0303e644ea28613cc80e4872119287744a2cd1587206a97a420c41"}
Jul 25 00:47:02.936896 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.936875    2574 generic.go:356] "Generic (PLEG): container finished" podID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerID="37a7ba0cb654baa7d579ce854621554071087bf5ac3cdee070e58aebe50364d4" exitCode=0
Jul 25 00:47:02.936994 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:02.936923    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd" event={"ID":"56fa2ac6-fdf6-4c8d-8507-9011420a4c73","Type":"ContainerDied","Data":"37a7ba0cb654baa7d579ce854621554071087bf5ac3cdee070e58aebe50364d4"}
Jul 25 00:47:04.093138 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.093108    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:47:04.132431 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.132403    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:47:04.166548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.166525    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:47:04.169815 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.169795    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:04.270905 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.270870    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhzc4\" (UniqueName: \"kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4\") pod \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") "
Jul 25 00:47:04.270905 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.270905    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util\") pod \"e4375d1c-f127-45b7-961c-96c979d96cb5\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.270956    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle\") pod \"da62d501-2e21-41c7-9d61-82a088ac7f46\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.270976    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util\") pod \"da62d501-2e21-41c7-9d61-82a088ac7f46\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.270996    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util\") pod \"ae843d58-08aa-4c34-9720-ddea4050b699\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271018    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle\") pod \"ae843d58-08aa-4c34-9720-ddea4050b699\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271062    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle\") pod \"e4375d1c-f127-45b7-961c-96c979d96cb5\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271097    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbtff\" (UniqueName: \"kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff\") pod \"ae843d58-08aa-4c34-9720-ddea4050b699\" (UID: \"ae843d58-08aa-4c34-9720-ddea4050b699\") "
Jul 25 00:47:04.271144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271126    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle\") pod \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") "
Jul 25 00:47:04.271495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271183    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9txjl\" (UniqueName: \"kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl\") pod \"da62d501-2e21-41c7-9d61-82a088ac7f46\" (UID: \"da62d501-2e21-41c7-9d61-82a088ac7f46\") "
Jul 25 00:47:04.271495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271213    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m49q\" (UniqueName: \"kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q\") pod \"e4375d1c-f127-45b7-961c-96c979d96cb5\" (UID: \"e4375d1c-f127-45b7-961c-96c979d96cb5\") "
Jul 25 00:47:04.271495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271238    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util\") pod \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\" (UID: \"56fa2ac6-fdf6-4c8d-8507-9011420a4c73\") "
Jul 25 00:47:04.271495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271478    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle" (OuterVolumeSpecName: "bundle") pod "da62d501-2e21-41c7-9d61-82a088ac7f46" (UID: "da62d501-2e21-41c7-9d61-82a088ac7f46"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.271738 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271727    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.272092 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.271949    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle" (OuterVolumeSpecName: "bundle") pod "ae843d58-08aa-4c34-9720-ddea4050b699" (UID: "ae843d58-08aa-4c34-9720-ddea4050b699"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.272295 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.272270    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle" (OuterVolumeSpecName: "bundle") pod "56fa2ac6-fdf6-4c8d-8507-9011420a4c73" (UID: "56fa2ac6-fdf6-4c8d-8507-9011420a4c73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.272527 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.272477    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle" (OuterVolumeSpecName: "bundle") pod "e4375d1c-f127-45b7-961c-96c979d96cb5" (UID: "e4375d1c-f127-45b7-961c-96c979d96cb5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.273601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.273568    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4" (OuterVolumeSpecName: "kube-api-access-mhzc4") pod "56fa2ac6-fdf6-4c8d-8507-9011420a4c73" (UID: "56fa2ac6-fdf6-4c8d-8507-9011420a4c73"). InnerVolumeSpecName "kube-api-access-mhzc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:04.274239 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.274212    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl" (OuterVolumeSpecName: "kube-api-access-9txjl") pod "da62d501-2e21-41c7-9d61-82a088ac7f46" (UID: "da62d501-2e21-41c7-9d61-82a088ac7f46"). InnerVolumeSpecName "kube-api-access-9txjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:04.274548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.274493    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff" (OuterVolumeSpecName: "kube-api-access-tbtff") pod "ae843d58-08aa-4c34-9720-ddea4050b699" (UID: "ae843d58-08aa-4c34-9720-ddea4050b699"). InnerVolumeSpecName "kube-api-access-tbtff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:04.274732 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.274716    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q" (OuterVolumeSpecName: "kube-api-access-6m49q") pod "e4375d1c-f127-45b7-961c-96c979d96cb5" (UID: "e4375d1c-f127-45b7-961c-96c979d96cb5"). InnerVolumeSpecName "kube-api-access-6m49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:04.277947 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.277923    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util" (OuterVolumeSpecName: "util") pod "ae843d58-08aa-4c34-9720-ddea4050b699" (UID: "ae843d58-08aa-4c34-9720-ddea4050b699"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.278244 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.278219    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util" (OuterVolumeSpecName: "util") pod "da62d501-2e21-41c7-9d61-82a088ac7f46" (UID: "da62d501-2e21-41c7-9d61-82a088ac7f46"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.278434 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.278415    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util" (OuterVolumeSpecName: "util") pod "56fa2ac6-fdf6-4c8d-8507-9011420a4c73" (UID: "56fa2ac6-fdf6-4c8d-8507-9011420a4c73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.303438 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.303411    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util" (OuterVolumeSpecName: "util") pod "e4375d1c-f127-45b7-961c-96c979d96cb5" (UID: "e4375d1c-f127-45b7-961c-96c979d96cb5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:04.372584 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372559    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372584 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372583    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbtff\" (UniqueName: \"kubernetes.io/projected/ae843d58-08aa-4c34-9720-ddea4050b699-kube-api-access-tbtff\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372595    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372604    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9txjl\" (UniqueName: \"kubernetes.io/projected/da62d501-2e21-41c7-9d61-82a088ac7f46-kube-api-access-9txjl\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372612    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m49q\" (UniqueName: \"kubernetes.io/projected/e4375d1c-f127-45b7-961c-96c979d96cb5-kube-api-access-6m49q\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372620    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372628    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzc4\" (UniqueName: \"kubernetes.io/projected/56fa2ac6-fdf6-4c8d-8507-9011420a4c73-kube-api-access-mhzc4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372636    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4375d1c-f127-45b7-961c-96c979d96cb5-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372644    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62d501-2e21-41c7-9d61-82a088ac7f46-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372652    2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-util\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.372730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.372659    2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae843d58-08aa-4c34-9720-ddea4050b699-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:04.948714 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.948675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm" event={"ID":"da62d501-2e21-41c7-9d61-82a088ac7f46","Type":"ContainerDied","Data":"edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1"}
Jul 25 00:47:04.948879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.948733    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edacadf2e64d762b0037434b4300bea0f1b854f696e0fb10e7583b68c62cbce1"
Jul 25 00:47:04.948879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.948682    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm"
Jul 25 00:47:04.950546 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.950526    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd"
Jul 25 00:47:04.950683 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.950531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd" event={"ID":"56fa2ac6-fdf6-4c8d-8507-9011420a4c73","Type":"ContainerDied","Data":"1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0"}
Jul 25 00:47:04.950683 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.950635    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e3ee6e7464e29356c96113055e0f9e04af202743a197a032cc9af76c80090f0"
Jul 25 00:47:04.952635 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.952618    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z"
Jul 25 00:47:04.952718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.952662    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z" event={"ID":"ae843d58-08aa-4c34-9720-ddea4050b699","Type":"ContainerDied","Data":"627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35"}
Jul 25 00:47:04.952718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.952706    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="627209ac556cda6cff01192dfde6c829779a138d24d273325d7c6401898ada35"
Jul 25 00:47:04.954815 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.954794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6" event={"ID":"e4375d1c-f127-45b7-961c-96c979d96cb5","Type":"ContainerDied","Data":"2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339"}
Jul 25 00:47:04.954901 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.954830    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f0608784cca891c5df7a70e4b45a9cf30e242991b7513265d8c0cdbd8b5a339"
Jul 25 00:47:04.954901 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:04.954832    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6"
Jul 25 00:47:09.593495 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:09.593455    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:09.654121 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:09.654081    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:11.316588 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:11.316552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:11.909021 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:11.908984    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:15.368328 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368290    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"]
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368686    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="util"
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368697    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="util"
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368707    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="util"
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368713    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="util"
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368720    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="pull"
Jul 25 00:47:15.368722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368726    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368732    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368737    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368747    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368752    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368760    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="util"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368765    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="util"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368775    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368780    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368788    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368794    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368800    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368805    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368815    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368819    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368825    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368829    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="pull"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368839    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="util"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368843    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="util"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368900    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="da62d501-2e21-41c7-9d61-82a088ac7f46" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368907    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae843d58-08aa-4c34-9720-ddea4050b699" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368915    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4375d1c-f127-45b7-961c-96c979d96cb5" containerName="extract"
Jul 25 00:47:15.368920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.368921    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fa2ac6-fdf6-4c8d-8507-9011420a4c73" containerName="extract"
Jul 25 00:47:15.372120 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.372102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:15.376912 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.376884    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bkzwv\""
Jul 25 00:47:15.382354 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.382328    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"]
Jul 25 00:47:15.468847 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.468818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5988\" (UniqueName: \"kubernetes.io/projected/029c4571-66d7-4d3e-8f56-99b61f141dd3-kube-api-access-w5988\") pod \"authorino-operator-75cc8d5cb7-z6xx8\" (UID: \"029c4571-66d7-4d3e-8f56-99b61f141dd3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:15.570056 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.570022    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5988\" (UniqueName: \"kubernetes.io/projected/029c4571-66d7-4d3e-8f56-99b61f141dd3-kube-api-access-w5988\") pod \"authorino-operator-75cc8d5cb7-z6xx8\" (UID: \"029c4571-66d7-4d3e-8f56-99b61f141dd3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:15.585020 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.584990    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5988\" (UniqueName: \"kubernetes.io/projected/029c4571-66d7-4d3e-8f56-99b61f141dd3-kube-api-access-w5988\") pod \"authorino-operator-75cc8d5cb7-z6xx8\" (UID: \"029c4571-66d7-4d3e-8f56-99b61f141dd3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:15.684732 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.684658    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:15.822605 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:15.822577    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"]
Jul 25 00:47:15.824987 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:15.824956    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029c4571_66d7_4d3e_8f56_99b61f141dd3.slice/crio-b8ee3c1f1b740c35a8e2e5283786aa204e243e5ac4459190450f9909b843c724 WatchSource:0}: Error finding container b8ee3c1f1b740c35a8e2e5283786aa204e243e5ac4459190450f9909b843c724: Status 404 returned error can't find the container with id b8ee3c1f1b740c35a8e2e5283786aa204e243e5ac4459190450f9909b843c724
Jul 25 00:47:16.017124 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:16.017091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8" event={"ID":"029c4571-66d7-4d3e-8f56-99b61f141dd3","Type":"ContainerStarted","Data":"b8ee3c1f1b740c35a8e2e5283786aa204e243e5ac4459190450f9909b843c724"}
Jul 25 00:47:16.743111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:16.743067    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:18.585034 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.584997    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"]
Jul 25 00:47:18.588742 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.588715    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:18.591103 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.591078    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 25 00:47:18.591203 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.591078    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2w8sr\""
Jul 25 00:47:18.597880 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.597859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"]
Jul 25 00:47:18.702275 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.702065    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j69c\" (UniqueName: \"kubernetes.io/projected/244aab7e-0470-452e-842b-0c972b85d65b-kube-api-access-7j69c\") pod \"dns-operator-controller-manager-864dd78c4c-jlvbp\" (UID: \"244aab7e-0470-452e-842b-0c972b85d65b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:18.785600 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.785565    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:18.803280 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.803228    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j69c\" (UniqueName: \"kubernetes.io/projected/244aab7e-0470-452e-842b-0c972b85d65b-kube-api-access-7j69c\") pod \"dns-operator-controller-manager-864dd78c4c-jlvbp\" (UID: \"244aab7e-0470-452e-842b-0c972b85d65b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:18.823971 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.823935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j69c\" (UniqueName: \"kubernetes.io/projected/244aab7e-0470-452e-842b-0c972b85d65b-kube-api-access-7j69c\") pod \"dns-operator-controller-manager-864dd78c4c-jlvbp\" (UID: \"244aab7e-0470-452e-842b-0c972b85d65b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:18.900107 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:18.900071    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:19.036473 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:19.036437    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8" event={"ID":"029c4571-66d7-4d3e-8f56-99b61f141dd3","Type":"ContainerStarted","Data":"1b69f12a27932b1a4cc41321c59c9d30b764f5c7462bf28b2ae41bc1a87aa124"}
Jul 25 00:47:19.036659 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:19.036538    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:19.046246 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:19.046222    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"]
Jul 25 00:47:19.048059 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:19.048036    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244aab7e_0470_452e_842b_0c972b85d65b.slice/crio-42964741b8583ee16288f0a897e77883e2a3a5fc01223d05bc6edadcd763f3f9 WatchSource:0}: Error finding container 42964741b8583ee16288f0a897e77883e2a3a5fc01223d05bc6edadcd763f3f9: Status 404 returned error can't find the container with id 42964741b8583ee16288f0a897e77883e2a3a5fc01223d05bc6edadcd763f3f9
Jul 25 00:47:19.056138 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:19.056095    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8" podStartSLOduration=1.8949828659999999 podStartE2EDuration="4.056080459s" totalImagesPullingTime="2.161097593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:19.053111643 +0000 UTC m=+693.764530043" watchObservedRunningTime="2026-07-25 00:47:19.056080459 +0000 UTC m=+693.767498857"
Jul 25 00:47:19.724301 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:19.724265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:20.045156 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:20.045063    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp" event={"ID":"244aab7e-0470-452e-842b-0c972b85d65b","Type":"ContainerStarted","Data":"42964741b8583ee16288f0a897e77883e2a3a5fc01223d05bc6edadcd763f3f9"}
Jul 25 00:47:20.554078 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:20.554036    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:21.052694 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.052660    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp" event={"ID":"244aab7e-0470-452e-842b-0c972b85d65b","Type":"ContainerStarted","Data":"6181c391c431d346e102a72e8bfcb19a97a88ed87028ecefe0a091c014c1314f"}
Jul 25 00:47:21.053037 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.052777    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:21.070808 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.070762    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp" podStartSLOduration=1.191221826 podStartE2EDuration="3.070750437s" totalImagesPullingTime="1.879528611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:21.067491104 +0000 UTC m=+695.778909504" watchObservedRunningTime="2026-07-25 00:47:21.070750437 +0000 UTC m=+695.782168836"
Jul 25 00:47:21.941971 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.941938    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7997d47658-s2gs2"]
Jul 25 00:47:21.946657 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.946631    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:21.948779 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.948751    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7997d47658-s2gs2"]
Jul 25 00:47:21.988040 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:21.987975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:22.035245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzpbx\" (UniqueName: \"kubernetes.io/projected/099ece9c-3e2d-403f-a44a-4e906b1800fc-kube-api-access-pzpbx\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-service-ca\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035406    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-oauth-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035679 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-trusted-ca-bundle\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.035679 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.035547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-oauth-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.096584 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.096548    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:22.116694 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.116652    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"]
Jul 25 00:47:22.120253 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.120236    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:22.122479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.122457    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-s62zz\""
Jul 25 00:47:22.132889 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.132862    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"]
Jul 25 00:47:22.136906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.136878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-trusted-ca-bundle\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137038 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.136964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-oauth-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137364 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137460 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzpbx\" (UniqueName: \"kubernetes.io/projected/099ece9c-3e2d-403f-a44a-4e906b1800fc-kube-api-access-pzpbx\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137550 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-service-ca\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137550 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137662 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-oauth-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.137919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.137893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-trusted-ca-bundle\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.138210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.138188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-oauth-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.138753 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.138633    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.138886 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.138752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/099ece9c-3e2d-403f-a44a-4e906b1800fc-service-ca\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.140088 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.140061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-oauth-config\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.140707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.140689    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/099ece9c-3e2d-403f-a44a-4e906b1800fc-console-serving-cert\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.152042 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.152013    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzpbx\" (UniqueName: \"kubernetes.io/projected/099ece9c-3e2d-403f-a44a-4e906b1800fc-kube-api-access-pzpbx\") pod \"console-7997d47658-s2gs2\" (UID: \"099ece9c-3e2d-403f-a44a-4e906b1800fc\") " pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.238761 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.238715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5lp\" (UniqueName: \"kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp\") pod \"limitador-operator-controller-manager-859d7b78bd-tg45k\" (UID: \"8fb7f660-d3a6-4cce-9baa-ee9338325745\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:22.257126 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.257093    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:22.340605 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.340106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5lp\" (UniqueName: \"kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp\") pod \"limitador-operator-controller-manager-859d7b78bd-tg45k\" (UID: \"8fb7f660-d3a6-4cce-9baa-ee9338325745\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:22.355816 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.355787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5lp\" (UniqueName: \"kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp\") pod \"limitador-operator-controller-manager-859d7b78bd-tg45k\" (UID: \"8fb7f660-d3a6-4cce-9baa-ee9338325745\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:22.417474 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.417447    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7997d47658-s2gs2"]
Jul 25 00:47:22.419689 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:22.419663    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099ece9c_3e2d_403f_a44a_4e906b1800fc.slice/crio-3eb60c63bab9003fc7bdfa2192ff8d24c9d503ce257718146594b13f0a2b5c0d WatchSource:0}: Error finding container 3eb60c63bab9003fc7bdfa2192ff8d24c9d503ce257718146594b13f0a2b5c0d: Status 404 returned error can't find the container with id 3eb60c63bab9003fc7bdfa2192ff8d24c9d503ce257718146594b13f0a2b5c0d
Jul 25 00:47:22.432143 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.432119    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:22.592653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.592627    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"]
Jul 25 00:47:22.594789 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:22.594763    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb7f660_d3a6_4cce_9baa_ee9338325745.slice/crio-e30a1cb1c1bcfba78c07bd5ce755c7eb2d27b2b5ac1c49497e4151f28b840452 WatchSource:0}: Error finding container e30a1cb1c1bcfba78c07bd5ce755c7eb2d27b2b5ac1c49497e4151f28b840452: Status 404 returned error can't find the container with id e30a1cb1c1bcfba78c07bd5ce755c7eb2d27b2b5ac1c49497e4151f28b840452
Jul 25 00:47:22.722176 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:22.722141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:23.066575 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:23.066537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k" event={"ID":"8fb7f660-d3a6-4cce-9baa-ee9338325745","Type":"ContainerStarted","Data":"e30a1cb1c1bcfba78c07bd5ce755c7eb2d27b2b5ac1c49497e4151f28b840452"}
Jul 25 00:47:23.068095 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:23.068071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7997d47658-s2gs2" event={"ID":"099ece9c-3e2d-403f-a44a-4e906b1800fc","Type":"ContainerStarted","Data":"c5092342d84d618b32e989616aa65c4562ca4484fca4e9d8e64e6d21d4cbdc3f"}
Jul 25 00:47:23.068224 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:23.068108    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7997d47658-s2gs2" event={"ID":"099ece9c-3e2d-403f-a44a-4e906b1800fc","Type":"ContainerStarted","Data":"3eb60c63bab9003fc7bdfa2192ff8d24c9d503ce257718146594b13f0a2b5c0d"}
Jul 25 00:47:23.088791 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:23.088731    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7997d47658-s2gs2" podStartSLOduration=2.088714761 podStartE2EDuration="2.088714761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:23.087145262 +0000 UTC m=+697.798563664" watchObservedRunningTime="2026-07-25 00:47:23.088714761 +0000 UTC m=+697.800133159"
Jul 25 00:47:23.227364 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:23.227322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:25.082221 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:25.082186    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k" event={"ID":"8fb7f660-d3a6-4cce-9baa-ee9338325745","Type":"ContainerStarted","Data":"f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01"}
Jul 25 00:47:25.082743 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:25.082328    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:25.098837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:25.098786    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k" podStartSLOduration=1.4247003440000001 podStartE2EDuration="3.098772953s" totalImagesPullingTime="1.674072609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:25.097178111 +0000 UTC m=+699.808596511" watchObservedRunningTime="2026-07-25 00:47:25.098772953 +0000 UTC m=+699.810191352"
Jul 25 00:47:26.368660 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.368624    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:26.892558 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.892524    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"]
Jul 25 00:47:26.896237 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.896221    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:26.899224 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.899201    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qvbjg\""
Jul 25 00:47:26.908713 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.908687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"]
Jul 25 00:47:26.991407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.991366    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:26.991407 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:26.991410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srch4\" (UniqueName: \"kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.092166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.092126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.092166 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.092170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srch4\" (UniqueName: \"kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.092573 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.092551    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.104879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.104852    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srch4\" (UniqueName: \"kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.207196 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.207170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:27.339601 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:27.339573    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"]
Jul 25 00:47:27.341150 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:27.341107    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730979b6_8cfa_4b33_935b_ea52cb8dbaa0.slice/crio-fa57f533e5552c632af214e04b806a8864668696e35ad1ee18f775fca534cf3b WatchSource:0}: Error finding container fa57f533e5552c632af214e04b806a8864668696e35ad1ee18f775fca534cf3b: Status 404 returned error can't find the container with id fa57f533e5552c632af214e04b806a8864668696e35ad1ee18f775fca534cf3b
Jul 25 00:47:28.108788 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:28.108748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6" event={"ID":"730979b6-8cfa-4b33-935b-ea52cb8dbaa0","Type":"ContainerStarted","Data":"fa57f533e5552c632af214e04b806a8864668696e35ad1ee18f775fca534cf3b"}
Jul 25 00:47:28.323420 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:28.323382    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:28.423549 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:28.423417    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:28.786859 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:28.786824    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:28.886942 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:28.886908    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:30.048088 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:30.048059    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z6xx8"
Jul 25 00:47:32.062226 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.062195    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jlvbp"
Jul 25 00:47:32.257648 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.257601    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:32.257844 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.257693    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:32.263505 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.263477    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:32.551712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.551660    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:32.649226 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:32.649191    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:33.143937 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.143905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6" event={"ID":"730979b6-8cfa-4b33-935b-ea52cb8dbaa0","Type":"ContainerStarted","Data":"360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144"}
Jul 25 00:47:33.147837 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.147811    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7997d47658-s2gs2"
Jul 25 00:47:33.166548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.166479    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6" podStartSLOduration=1.788446604 podStartE2EDuration="7.166463415s" totalImagesPullingTime="5.378016811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:33.163203452 +0000 UTC m=+707.874621852" watchObservedRunningTime="2026-07-25 00:47:33.166463415 +0000 UTC m=+707.877881813"
Jul 25 00:47:33.212144 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.212102    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:47:33.316242 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.316168    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:33.518078 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.518027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:33.615943 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:33.615904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:34.149534 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:34.149480    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:35.569794 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:35.569756    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:35.623544 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:35.623460    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:36.095734 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:36.095704    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:36.369057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:36.368963    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:36.469971 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:36.469935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:36.729860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:36.729825    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:36.829826 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:36.829791    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:40.931928 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:40.931841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:40.983920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:40.983888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:45.158188 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:45.158155    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:46.035272 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.035239    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"]
Jul 25 00:47:46.043249 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.043151    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.055134 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.055104    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"]
Jul 25 00:47:46.071977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.071946    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.072153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.071984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8v7b\" (UniqueName: \"kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.173442 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.173411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.173442 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.173446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8v7b\" (UniqueName: \"kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.173984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.173888    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.184165 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.184141    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8v7b\" (UniqueName: \"kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.356464 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.356375    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:46.514534 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.514492    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"]
Jul 25 00:47:46.516559 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:46.516498    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb4d816_ae28_45d6_bedb_92f156988002.slice/crio-1680bbbea54e1ce7bad41299f894a547b90205345c01f1cd7f36195944e7fae5 WatchSource:0}: Error finding container 1680bbbea54e1ce7bad41299f894a547b90205345c01f1cd7f36195944e7fae5: Status 404 returned error can't find the container with id 1680bbbea54e1ce7bad41299f894a547b90205345c01f1cd7f36195944e7fae5
Jul 25 00:47:46.840881 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.840833    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"]
Jul 25 00:47:46.845052 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.845023    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"]
Jul 25 00:47:46.856959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.856931    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"]
Jul 25 00:47:46.857182 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.857158    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" containerName="manager" containerID="cri-o://360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144" gracePeriod=2
Jul 25 00:47:46.859602 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.859445    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:46.862388 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.862365    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"]
Jul 25 00:47:46.871838 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.871813    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:47:46.872337 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872314    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" containerName="manager"
Jul 25 00:47:46.872337 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872331    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" containerName="manager"
Jul 25 00:47:46.872426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872341    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" containerName="manager"
Jul 25 00:47:46.872426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872347    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" containerName="manager"
Jul 25 00:47:46.872426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872406    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" containerName="manager"
Jul 25 00:47:46.872426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.872414    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" containerName="manager"
Jul 25 00:47:46.876111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.875983    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:46.876111 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.876026    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"]
Jul 25 00:47:46.876362 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.876211    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" containerName="manager" containerID="cri-o://f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01" gracePeriod=2
Jul 25 00:47:46.884354 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.883997    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:47:46.888367 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.888344    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:47:46.888461 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.888426    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:46.894343 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.894323    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"]
Jul 25 00:47:46.897371 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.897347    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:47:46.906504 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.906467    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:46.909391 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.909357    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:46.911579 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.911555    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"]
Jul 25 00:47:46.912180 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.912159    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" containerName="manager"
Jul 25 00:47:46.912247 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.912184    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" containerName="manager"
Jul 25 00:47:46.912349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.912335    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" containerName="manager"
Jul 25 00:47:46.915998 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.915978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:46.920833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.920805    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"]
Jul 25 00:47:46.932098 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.932055    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:46.934265 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.934055    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:46.935985 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.935958    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:46.937756 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:46.937734    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:46.983564 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.983533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt55t\" (UniqueName: \"kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:46.983676 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.983575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:46.983676 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.983598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:46.983885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.983685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fznt7\" (UniqueName: \"kubernetes.io/projected/43a44b7a-a867-4fc5-ad61-e3f536fc5570-kube-api-access-fznt7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvwgv\" (UID: \"43a44b7a-a867-4fc5-ad61-e3f536fc5570\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:46.983885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:46.983732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqw22\" (UniqueName: \"kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.085296 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fznt7\" (UniqueName: \"kubernetes.io/projected/43a44b7a-a867-4fc5-ad61-e3f536fc5570-kube-api-access-fznt7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvwgv\" (UID: \"43a44b7a-a867-4fc5-ad61-e3f536fc5570\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:47.085446 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqw22\" (UniqueName: \"kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.085446 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt55t\" (UniqueName: \"kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:47.085446 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:47.085446 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.085890 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.085890 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.085877    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:47.093376 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.093316    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqw22\" (UniqueName: \"kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7pxkc\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.093376 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.093366    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt55t\" (UniqueName: \"kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hh7bx\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:47.094139 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.094119    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fznt7\" (UniqueName: \"kubernetes.io/projected/43a44b7a-a867-4fc5-ad61-e3f536fc5570-kube-api-access-fznt7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvwgv\" (UID: \"43a44b7a-a867-4fc5-ad61-e3f536fc5570\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:47.142120 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.142096    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.144327 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.144309    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.144495 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.144476    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.146447 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.146422    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.148449 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.148431    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.150440 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.150419    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.186567 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.186545    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srch4\" (UniqueName: \"kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4\") pod \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") "
Jul 25 00:47:47.186958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.186584    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume\") pod \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\" (UID: \"730979b6-8cfa-4b33-935b-ea52cb8dbaa0\") "
Jul 25 00:47:47.186958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.186674    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md5lp\" (UniqueName: \"kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp\") pod \"8fb7f660-d3a6-4cce-9baa-ee9338325745\" (UID: \"8fb7f660-d3a6-4cce-9baa-ee9338325745\") "
Jul 25 00:47:47.187079 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.187022    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "730979b6-8cfa-4b33-935b-ea52cb8dbaa0" (UID: "730979b6-8cfa-4b33-935b-ea52cb8dbaa0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:47.188718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.188693    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp" (OuterVolumeSpecName: "kube-api-access-md5lp") pod "8fb7f660-d3a6-4cce-9baa-ee9338325745" (UID: "8fb7f660-d3a6-4cce-9baa-ee9338325745"). InnerVolumeSpecName "kube-api-access-md5lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:47.188833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.188726    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4" (OuterVolumeSpecName: "kube-api-access-srch4") pod "730979b6-8cfa-4b33-935b-ea52cb8dbaa0" (UID: "730979b6-8cfa-4b33-935b-ea52cb8dbaa0"). InnerVolumeSpecName "kube-api-access-srch4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:47.228701 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.228675    2574 generic.go:356] "Generic (PLEG): container finished" podID="8fb7f660-d3a6-4cce-9baa-ee9338325745" containerID="f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01" exitCode=0
Jul 25 00:47:47.228817 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.228722    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.228817 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.228771    2574 scope.go:117] "RemoveContainer" containerID="f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01"
Jul 25 00:47:47.230378 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.230355    2574 generic.go:356] "Generic (PLEG): container finished" podID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" containerID="360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144" exitCode=0
Jul 25 00:47:47.230463 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.230409    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.230932 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.230896    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.232709 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.232616    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls" container="manager"
Jul 25 00:47:47.234943 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.233420    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.235348 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.235321    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.237402 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.237375    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:47.239499 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.239470    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.243003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.242893    2574 scope.go:117] "RemoveContainer" containerID="f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01"
Jul 25 00:47:47.243311 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.243232    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01\": container with ID starting with f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01 not found: ID does not exist" containerID="f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01"
Jul 25 00:47:47.243311 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.243268    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01"} err="failed to get container status \"f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01\": rpc error: code = NotFound desc = could not find container \"f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01\": container with ID starting with f882bcc8ba246f85d100a0ee9ab18f9f5b06e605badb72104194d4bb4969da01 not found: ID does not exist"
Jul 25 00:47:47.243311 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.243291    2574 scope.go:117] "RemoveContainer" containerID="360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144"
Jul 25 00:47:47.246460 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.246437    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.248431 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.248410    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.249918 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.249893    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:47.251839 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.251816    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tg45k\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tg45k"
Jul 25 00:47:47.253611 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.253575    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:47.256336 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.256304    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fwpk6\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fwpk6"
Jul 25 00:47:47.256642 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.256614    2574 scope.go:117] "RemoveContainer" containerID="360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144"
Jul 25 00:47:47.257067 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:47.257042    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144\": container with ID starting with 360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144 not found: ID does not exist" containerID="360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144"
Jul 25 00:47:47.257156 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.257077    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144"} err="failed to get container status \"360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144\": rpc error: code = NotFound desc = could not find container \"360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144\": container with ID starting with 360e70931ca6316685d6e322e7e5f2364e03bcb34b7663ca725b42d38bae6144 not found: ID does not exist"
Jul 25 00:47:47.274677 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.274656    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:47.280225 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.280211    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:47.286855 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.286834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:47.287458 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.287434    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md5lp\" (UniqueName: \"kubernetes.io/projected/8fb7f660-d3a6-4cce-9baa-ee9338325745-kube-api-access-md5lp\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:47.287553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.287464    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srch4\" (UniqueName: \"kubernetes.io/projected/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-kube-api-access-srch4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:47.287553 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.287482    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/730979b6-8cfa-4b33-935b-ea52cb8dbaa0-extensions-socket-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:47.459343 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.459316    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:47:47.462378 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:47.462354    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1842d9_ef18_4efc_a7e0_deabb39687e6.slice/crio-e92d17625ecc6e82785bc221c28ef95847ca9359ef6ed80323b0e79de9fc7b40 WatchSource:0}: Error finding container e92d17625ecc6e82785bc221c28ef95847ca9359ef6ed80323b0e79de9fc7b40: Status 404 returned error can't find the container with id e92d17625ecc6e82785bc221c28ef95847ca9359ef6ed80323b0e79de9fc7b40
Jul 25 00:47:47.488018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.487971    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:47:47.491734 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.491714    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"]
Jul 25 00:47:47.500885 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:47.500861    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd6cbae_030e_4aab_a0d4_896cdbaf2359.slice/crio-4137be1fd56aa8223d3e13a79879fe1c26f8e2e8e2845e726e4bdd55de785a16 WatchSource:0}: Error finding container 4137be1fd56aa8223d3e13a79879fe1c26f8e2e8e2845e726e4bdd55de785a16: Status 404 returned error can't find the container with id 4137be1fd56aa8223d3e13a79879fe1c26f8e2e8e2845e726e4bdd55de785a16
Jul 25 00:47:47.501094 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:47:47.501071    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a44b7a_a867_4fc5_ad61_e3f536fc5570.slice/crio-6cb5d1b100624b786b7f1cd0fcbaece957b1797e8053defb4fc3b186e0351177 WatchSource:0}: Error finding container 6cb5d1b100624b786b7f1cd0fcbaece957b1797e8053defb4fc3b186e0351177: Status 404 returned error can't find the container with id 6cb5d1b100624b786b7f1cd0fcbaece957b1797e8053defb4fc3b186e0351177
Jul 25 00:47:47.938367 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.938324    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="730979b6-8cfa-4b33-935b-ea52cb8dbaa0" path="/var/lib/kubelet/pods/730979b6-8cfa-4b33-935b-ea52cb8dbaa0/volumes"
Jul 25 00:47:47.938968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:47.938943    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb7f660-d3a6-4cce-9baa-ee9338325745" path="/var/lib/kubelet/pods/8fb7f660-d3a6-4cce-9baa-ee9338325745/volumes"
Jul 25 00:47:48.248612 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.248563    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" containerName="manager" containerID="cri-o://14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7" gracePeriod=2
Jul 25 00:47:48.250320 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.250288    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" event={"ID":"5b1842d9-ef18-4efc-a7e0-deabb39687e6","Type":"ContainerStarted","Data":"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"}
Jul 25 00:47:48.250384 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.250347    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" event={"ID":"5b1842d9-ef18-4efc-a7e0-deabb39687e6","Type":"ContainerStarted","Data":"e92d17625ecc6e82785bc221c28ef95847ca9359ef6ed80323b0e79de9fc7b40"}
Jul 25 00:47:48.250600 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.250580    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:48.252899 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.252874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv" event={"ID":"43a44b7a-a867-4fc5-ad61-e3f536fc5570","Type":"ContainerStarted","Data":"36bc70a7321c8176767c04868b4031917492f7380878e1c3444e01dd7028bb90"}
Jul 25 00:47:48.253024 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.252919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv" event={"ID":"43a44b7a-a867-4fc5-ad61-e3f536fc5570","Type":"ContainerStarted","Data":"6cb5d1b100624b786b7f1cd0fcbaece957b1797e8053defb4fc3b186e0351177"}
Jul 25 00:47:48.253024 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.252942    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:48.254860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.254812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" event={"ID":"7cd6cbae-030e-4aab-a0d4-896cdbaf2359","Type":"ContainerStarted","Data":"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"}
Jul 25 00:47:48.254968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.254876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" event={"ID":"7cd6cbae-030e-4aab-a0d4-896cdbaf2359","Type":"ContainerStarted","Data":"4137be1fd56aa8223d3e13a79879fe1c26f8e2e8e2845e726e4bdd55de785a16"}
Jul 25 00:47:48.254968 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.254934    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:48.276615 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.276569    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" podStartSLOduration=2.276550508 podStartE2EDuration="2.276550508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:48.273120978 +0000 UTC m=+722.984539376" watchObservedRunningTime="2026-07-25 00:47:48.276550508 +0000 UTC m=+722.987968909"
Jul 25 00:47:48.297017 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.296963    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" podStartSLOduration=2.29694871 podStartE2EDuration="2.29694871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:48.294502064 +0000 UTC m=+723.005920466" watchObservedRunningTime="2026-07-25 00:47:48.29694871 +0000 UTC m=+723.008367113"
Jul 25 00:47:48.309901 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.309852    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv" podStartSLOduration=2.309833882 podStartE2EDuration="2.309833882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:47:48.309441673 +0000 UTC m=+723.020860097" watchObservedRunningTime="2026-07-25 00:47:48.309833882 +0000 UTC m=+723.021252282"
Jul 25 00:47:48.499532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.499453    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:48.501604 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:48.501573    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:48.600154 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.600121    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume\") pod \"cfb4d816-ae28-45d6-bedb-92f156988002\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") "
Jul 25 00:47:48.600318 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.600264    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8v7b\" (UniqueName: \"kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b\") pod \"cfb4d816-ae28-45d6-bedb-92f156988002\" (UID: \"cfb4d816-ae28-45d6-bedb-92f156988002\") "
Jul 25 00:47:48.600457 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.600434    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cfb4d816-ae28-45d6-bedb-92f156988002" (UID: "cfb4d816-ae28-45d6-bedb-92f156988002"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:48.602344 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.602319    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b" (OuterVolumeSpecName: "kube-api-access-d8v7b") pod "cfb4d816-ae28-45d6-bedb-92f156988002" (UID: "cfb4d816-ae28-45d6-bedb-92f156988002"). InnerVolumeSpecName "kube-api-access-d8v7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:48.701221 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.701184    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8v7b\" (UniqueName: \"kubernetes.io/projected/cfb4d816-ae28-45d6-bedb-92f156988002-kube-api-access-d8v7b\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:48.701221 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:48.701216    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb4d816-ae28-45d6-bedb-92f156988002-extensions-socket-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:49.261861 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.261827    2574 generic.go:356] "Generic (PLEG): container finished" podID="cfb4d816-ae28-45d6-bedb-92f156988002" containerID="14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7" exitCode=0
Jul 25 00:47:49.262329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.261871    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:49.262329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.261921    2574 scope.go:117] "RemoveContainer" containerID="14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7"
Jul 25 00:47:49.264069 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:49.264037    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:49.271486 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.271467    2574 scope.go:117] "RemoveContainer" containerID="14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7"
Jul 25 00:47:49.271755 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:49.271737    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7\": container with ID starting with 14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7 not found: ID does not exist" containerID="14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7"
Jul 25 00:47:49.271805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.271763    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7"} err="failed to get container status \"14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7\": rpc error: code = NotFound desc = could not find container \"14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7\": container with ID starting with 14d010c9fe0c4e486920428bc08ec2f36e00469ca440fa8b81979cd3b9e8b2f7 not found: ID does not exist"
Jul 25 00:47:49.272315 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:49.272298    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kl4ls\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kl4ls"
Jul 25 00:47:49.933532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:49.933488    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfb4d816-ae28-45d6-bedb-92f156988002" path="/var/lib/kubelet/pods/cfb4d816-ae28-45d6-bedb-92f156988002/volumes"
Jul 25 00:47:50.318213 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:50.318174    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:50.416879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:50.416843    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:50.961673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:50.961638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:51.022291 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:51.022256    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:54.730050 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:54.730009    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:54.830822 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:54.830785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:58.237036 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.236992    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bbbf654fd-mdw7x" podUID="796d64eb-47da-4335-9055-072f00bf940c" containerName="console" containerID="cri-o://1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432" gracePeriod=15
Jul 25 00:47:58.493564 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.493490    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:47:58.593786 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.593749    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.593988 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.593823    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.593988 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.593914    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.593988 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.593943    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.593988 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.593979    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.594204 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594023    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmfh7\" (UniqueName: \"kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.594204 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594080    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config\") pod \"796d64eb-47da-4335-9055-072f00bf940c\" (UID: \"796d64eb-47da-4335-9055-072f00bf940c\") "
Jul 25 00:47:58.594336 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594284    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config" (OuterVolumeSpecName: "console-config") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:47:58.594396 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594354    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:47:58.594396 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594367    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:47:58.594505 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594398    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca" (OuterVolumeSpecName: "service-ca") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:47:58.594505 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594489    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.594628 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594540    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-oauth-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.594628 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594558    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-console-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.594628 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.594567    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/796d64eb-47da-4335-9055-072f00bf940c-trusted-ca-bundle\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.596245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.596218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:47:58.596245 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.596231    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7" (OuterVolumeSpecName: "kube-api-access-pmfh7") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "kube-api-access-pmfh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:58.596359 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.596264    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "796d64eb-47da-4335-9055-072f00bf940c" (UID: "796d64eb-47da-4335-9055-072f00bf940c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:47:58.695860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.695819    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-serving-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.695860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.695857    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmfh7\" (UniqueName: \"kubernetes.io/projected/796d64eb-47da-4335-9055-072f00bf940c-kube-api-access-pmfh7\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.696140 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.695874    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/796d64eb-47da-4335-9055-072f00bf940c-console-oauth-config\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:58.721603 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.721566    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:58.822744 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:58.822642    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:47:59.264730 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.264696    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvwgv"
Jul 25 00:47:59.265217 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.264771    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:59.265217 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.265114    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:47:59.332630 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.332599    2574 generic.go:356] "Generic (PLEG): container finished" podID="796d64eb-47da-4335-9055-072f00bf940c" containerID="1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432" exitCode=0
Jul 25 00:47:59.332799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.332665    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbbf654fd-mdw7x"
Jul 25 00:47:59.332799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.332685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbf654fd-mdw7x" event={"ID":"796d64eb-47da-4335-9055-072f00bf940c","Type":"ContainerDied","Data":"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"}
Jul 25 00:47:59.332799 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.332779    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbbf654fd-mdw7x" event={"ID":"796d64eb-47da-4335-9055-072f00bf940c","Type":"ContainerDied","Data":"9627339fa680ee3c1696d94db26acdef622d62b8292f71e5c8531a65d6b73b04"}
Jul 25 00:47:59.332915 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.332807    2574 scope.go:117] "RemoveContainer" containerID="1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"
Jul 25 00:47:59.344060 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.344038    2574 scope.go:117] "RemoveContainer" containerID="1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"
Jul 25 00:47:59.344360 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:47:59.344340    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432\": container with ID starting with 1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432 not found: ID does not exist" containerID="1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"
Jul 25 00:47:59.344426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.344370    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432"} err="failed to get container status \"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432\": rpc error: code = NotFound desc = could not find container \"1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432\": container with ID starting with 1cefd1f1398eee1c71e8870f5ea92d6113f49747ee7dd300c58caf93626b5432 not found: ID does not exist"
Jul 25 00:47:59.355530 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.355484    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:47:59.355746 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.355723    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" podUID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" containerName="manager" containerID="cri-o://15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f" gracePeriod=10
Jul 25 00:47:59.365631 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.365600    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:47:59.374067 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.374040    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bbbf654fd-mdw7x"]
Jul 25 00:47:59.615106 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.615079    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:47:59.690590 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.690545    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:47:59.691194 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691172    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" containerName="manager"
Jul 25 00:47:59.691194 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691195    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" containerName="manager"
Jul 25 00:47:59.691345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691216    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796d64eb-47da-4335-9055-072f00bf940c" containerName="console"
Jul 25 00:47:59.691345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691222    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="796d64eb-47da-4335-9055-072f00bf940c" containerName="console"
Jul 25 00:47:59.691345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691322    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="796d64eb-47da-4335-9055-072f00bf940c" containerName="console"
Jul 25 00:47:59.691345 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.691336    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" containerName="manager"
Jul 25 00:47:59.694649 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.694632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.704785 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.704756    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqw22\" (UniqueName: \"kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22\") pod \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") "
Jul 25 00:47:59.704920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.704906    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume\") pod \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\" (UID: \"7cd6cbae-030e-4aab-a0d4-896cdbaf2359\") "
Jul 25 00:47:59.705243 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.705220    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7cd6cbae-030e-4aab-a0d4-896cdbaf2359" (UID: "7cd6cbae-030e-4aab-a0d4-896cdbaf2359"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:47:59.707015 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.706989    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22" (OuterVolumeSpecName: "kube-api-access-wqw22") pod "7cd6cbae-030e-4aab-a0d4-896cdbaf2359" (UID: "7cd6cbae-030e-4aab-a0d4-896cdbaf2359"). InnerVolumeSpecName "kube-api-access-wqw22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:47:59.710433 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.710410    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:47:59.805707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.805619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.805707 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.805658    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn6hs\" (UniqueName: \"kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.805894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.805769    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-extensions-socket-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:59.805894 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.805785    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqw22\" (UniqueName: \"kubernetes.io/projected/7cd6cbae-030e-4aab-a0d4-896cdbaf2359-kube-api-access-wqw22\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:47:59.906919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.906885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.906919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.906925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn6hs\" (UniqueName: \"kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.907267 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.907245    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.914673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.914650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn6hs\" (UniqueName: \"kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs\") pod \"kuadrant-operator-controller-manager-86878c769f-zp6sj\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:47:59.932959 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:47:59.932932    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="796d64eb-47da-4335-9055-072f00bf940c" path="/var/lib/kubelet/pods/796d64eb-47da-4335-9055-072f00bf940c/volumes"
Jul 25 00:48:00.004884 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.004850    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:48:00.143699 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.143669    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:48:00.145622 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:48:00.145592    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be15fde_945e_44ba_847f_2cab3ee5b3cb.slice/crio-561ed35a41d213858f84b5954c36fae8031c6430a4aa1f0a7b085a4d472632c8 WatchSource:0}: Error finding container 561ed35a41d213858f84b5954c36fae8031c6430a4aa1f0a7b085a4d472632c8: Status 404 returned error can't find the container with id 561ed35a41d213858f84b5954c36fae8031c6430a4aa1f0a7b085a4d472632c8
Jul 25 00:48:00.341449 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.341347    2574 generic.go:356] "Generic (PLEG): container finished" podID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" containerID="15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f" exitCode=0
Jul 25 00:48:00.341449 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.341398    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"
Jul 25 00:48:00.341449 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.341430    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" event={"ID":"7cd6cbae-030e-4aab-a0d4-896cdbaf2359","Type":"ContainerDied","Data":"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"}
Jul 25 00:48:00.342027 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.341498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc" event={"ID":"7cd6cbae-030e-4aab-a0d4-896cdbaf2359","Type":"ContainerDied","Data":"4137be1fd56aa8223d3e13a79879fe1c26f8e2e8e2845e726e4bdd55de785a16"}
Jul 25 00:48:00.342027 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.341551    2574 scope.go:117] "RemoveContainer" containerID="15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"
Jul 25 00:48:00.343447 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.343391    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" event={"ID":"1be15fde-945e-44ba-847f-2cab3ee5b3cb","Type":"ContainerStarted","Data":"c211e02bf36b7387a00e735358b33edd377a21fd057e987676d9aba24f801dd8"}
Jul 25 00:48:00.343447 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.343441    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" event={"ID":"1be15fde-945e-44ba-847f-2cab3ee5b3cb","Type":"ContainerStarted","Data":"561ed35a41d213858f84b5954c36fae8031c6430a4aa1f0a7b085a4d472632c8"}
Jul 25 00:48:00.343867 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.343470    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:48:00.351606 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.351582    2574 scope.go:117] "RemoveContainer" containerID="15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"
Jul 25 00:48:00.351902 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:00.351874    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f\": container with ID starting with 15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f not found: ID does not exist" containerID="15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"
Jul 25 00:48:00.352011 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.351903    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f"} err="failed to get container status \"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f\": rpc error: code = NotFound desc = could not find container \"15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f\": container with ID starting with 15aeefde7e4b2ca76afcef6301ce780657fa8a9f7ade71bb62233437f90e491f not found: ID does not exist"
Jul 25 00:48:00.372875 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.372847    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:48:00.375185 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.375164    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7pxkc"]
Jul 25 00:48:00.391057 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:00.391009    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" podStartSLOduration=1.390997479 podStartE2EDuration="1.390997479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:48:00.388954033 +0000 UTC m=+735.100372432" watchObservedRunningTime="2026-07-25 00:48:00.390997479 +0000 UTC m=+735.102415878"
Jul 25 00:48:01.933909 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:01.933874    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cd6cbae-030e-4aab-a0d4-896cdbaf2359" path="/var/lib/kubelet/pods/7cd6cbae-030e-4aab-a0d4-896cdbaf2359/volumes"
Jul 25 00:48:02.517504 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:02.517468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:02.570532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:02.570481    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:11.353555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.353529    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:48:11.403276 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.403237    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:48:11.403474 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.403429    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" podUID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" containerName="manager" containerID="cri-o://4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832" gracePeriod=10
Jul 25 00:48:11.661885 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.661861    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:48:11.812266 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.812237    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume\") pod \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") "
Jul 25 00:48:11.812467 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.812329    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt55t\" (UniqueName: \"kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t\") pod \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\" (UID: \"5b1842d9-ef18-4efc-a7e0-deabb39687e6\") "
Jul 25 00:48:11.812660 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.812634    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b1842d9-ef18-4efc-a7e0-deabb39687e6" (UID: "5b1842d9-ef18-4efc-a7e0-deabb39687e6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:48:11.814370 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.814345    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t" (OuterVolumeSpecName: "kube-api-access-zt55t") pod "5b1842d9-ef18-4efc-a7e0-deabb39687e6" (UID: "5b1842d9-ef18-4efc-a7e0-deabb39687e6"). InnerVolumeSpecName "kube-api-access-zt55t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:48:11.913503 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.913426    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1842d9-ef18-4efc-a7e0-deabb39687e6-extensions-socket-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:11.913503 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:11.913450    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt55t\" (UniqueName: \"kubernetes.io/projected/5b1842d9-ef18-4efc-a7e0-deabb39687e6-kube-api-access-zt55t\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:12.419966 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.419933    2574 generic.go:356] "Generic (PLEG): container finished" podID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" containerID="4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832" exitCode=0
Jul 25 00:48:12.420479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.419986    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"
Jul 25 00:48:12.420479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.420013    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" event={"ID":"5b1842d9-ef18-4efc-a7e0-deabb39687e6","Type":"ContainerDied","Data":"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"}
Jul 25 00:48:12.420479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.420071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx" event={"ID":"5b1842d9-ef18-4efc-a7e0-deabb39687e6","Type":"ContainerDied","Data":"e92d17625ecc6e82785bc221c28ef95847ca9359ef6ed80323b0e79de9fc7b40"}
Jul 25 00:48:12.420479 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.420095    2574 scope.go:117] "RemoveContainer" containerID="4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"
Jul 25 00:48:12.429597 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.429575    2574 scope.go:117] "RemoveContainer" containerID="4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"
Jul 25 00:48:12.429842 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:12.429813    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832\": container with ID starting with 4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832 not found: ID does not exist" containerID="4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"
Jul 25 00:48:12.429892 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.429840    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832"} err="failed to get container status \"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832\": rpc error: code = NotFound desc = could not find container \"4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832\": container with ID starting with 4e5504dd96b97d96d211d2a54d9ddbba27a5f25f21fd7126b9830ecf052a8832 not found: ID does not exist"
Jul 25 00:48:12.439725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.439704    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:48:12.445835 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:12.445808    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hh7bx"]
Jul 25 00:48:13.932940 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:13.932899    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" path="/var/lib/kubelet/pods/5b1842d9-ef18-4efc-a7e0-deabb39687e6/volumes"
Jul 25 00:48:15.739299 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:15.739265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:15.945673 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:15.945634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:28.946053 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:28.946012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:28.997751 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:28.997723    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:29.870301 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.870257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:29.878314 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.878283    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:29.878736 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.878722    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" containerName="manager"
Jul 25 00:48:29.878793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.878738    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" containerName="manager"
Jul 25 00:48:29.878833 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.878817    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b1842d9-ef18-4efc-a7e0-deabb39687e6" containerName="manager"
Jul 25 00:48:29.883329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.883311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:29.885778 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.885700    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 25 00:48:29.886631 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.886614    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9qz72\""
Jul 25 00:48:29.888927 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.888906    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:29.966580 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.966544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gt4\" (UniqueName: \"kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:29.966999 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.966608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:29.969863 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.969832    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:29.977723 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:29.977689    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:30.025797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.025770    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:30.067355 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.067323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gt4\" (UniqueName: \"kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:30.067472 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.067380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:30.068174 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.068147    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:30.075043 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.075022    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gt4\" (UniqueName: \"kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4\") pod \"limitador-limitador-64c94fd64f-z8gmw\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:30.195899 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.195807    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:30.337127 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.337067    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:30.339336 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:48:30.339303    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52058f8_0dd4_4bf1_8c6b_cbd4da4d8fdd.slice/crio-ad007e22c8f8e5959ec41a7ee82ff8325427f1c23da30445fdcb5471c2f561b2 WatchSource:0}: Error finding container ad007e22c8f8e5959ec41a7ee82ff8325427f1c23da30445fdcb5471c2f561b2: Status 404 returned error can't find the container with id ad007e22c8f8e5959ec41a7ee82ff8325427f1c23da30445fdcb5471c2f561b2
Jul 25 00:48:30.526433 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.526392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" event={"ID":"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd","Type":"ContainerStarted","Data":"ad007e22c8f8e5959ec41a7ee82ff8325427f1c23da30445fdcb5471c2f561b2"}
Jul 25 00:48:30.670236 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.670194    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:30.675361 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.675337    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:30.677860 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.677834    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nn7tv\""
Jul 25 00:48:30.680488 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.680463    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:30.699914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.699889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:30.754131 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.754105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:30.774219 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.774196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ms2m\" (UniqueName: \"kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m\") pod \"authorino-58cd955675-xpt7v\" (UID: \"ba3fb184-2a54-4b53-8591-bf306fd0bba6\") " pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:30.875639 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.875558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ms2m\" (UniqueName: \"kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m\") pod \"authorino-58cd955675-xpt7v\" (UID: \"ba3fb184-2a54-4b53-8591-bf306fd0bba6\") " pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:30.883105 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.883076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ms2m\" (UniqueName: \"kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m\") pod \"authorino-58cd955675-xpt7v\" (UID: \"ba3fb184-2a54-4b53-8591-bf306fd0bba6\") " pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:30.987198 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:30.987167    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:31.173226 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:31.173191    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:31.174728 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:48:31.174697    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba3fb184_2a54_4b53_8591_bf306fd0bba6.slice/crio-2ec62b35fe533d55d694d29f8bea818bc1540f5a888e3d0ba06683e20dda385d WatchSource:0}: Error finding container 2ec62b35fe533d55d694d29f8bea818bc1540f5a888e3d0ba06683e20dda385d: Status 404 returned error can't find the container with id 2ec62b35fe533d55d694d29f8bea818bc1540f5a888e3d0ba06683e20dda385d
Jul 25 00:48:31.537009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:31.536967    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xpt7v" event={"ID":"ba3fb184-2a54-4b53-8591-bf306fd0bba6","Type":"ContainerStarted","Data":"2ec62b35fe533d55d694d29f8bea818bc1540f5a888e3d0ba06683e20dda385d"}
Jul 25 00:48:34.304971 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:34.304927    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:35.566960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.566926    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xpt7v" event={"ID":"ba3fb184-2a54-4b53-8591-bf306fd0bba6","Type":"ContainerStarted","Data":"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"}
Jul 25 00:48:35.567409 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.566942    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xpt7v" podUID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" containerName="authorino" containerID="cri-o://9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc" gracePeriod=30
Jul 25 00:48:35.568619 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.568596    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" event={"ID":"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd","Type":"ContainerStarted","Data":"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"}
Jul 25 00:48:35.568756 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.568653    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:35.585804 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.585768    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xpt7v" podStartSLOduration=1.849105035 podStartE2EDuration="5.585755364s" totalImagesPullingTime="3.736650329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:48:35.583225075 +0000 UTC m=+770.294643485" watchObservedRunningTime="2026-07-25 00:48:35.585755364 +0000 UTC m=+770.297173763"
Jul 25 00:48:35.598523 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.598470    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" podStartSLOduration=2.038857062 podStartE2EDuration="6.598456871s" totalImagesPullingTime="4.559599809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:48:35.596044161 +0000 UTC m=+770.307462560" watchObservedRunningTime="2026-07-25 00:48:35.598456871 +0000 UTC m=+770.309875269"
Jul 25 00:48:35.823310 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.823251    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:35.922438 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.922409    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ms2m\" (UniqueName: \"kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m\") pod \"ba3fb184-2a54-4b53-8591-bf306fd0bba6\" (UID: \"ba3fb184-2a54-4b53-8591-bf306fd0bba6\") "
Jul 25 00:48:35.924389 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:35.924366    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m" (OuterVolumeSpecName: "kube-api-access-8ms2m") pod "ba3fb184-2a54-4b53-8591-bf306fd0bba6" (UID: "ba3fb184-2a54-4b53-8591-bf306fd0bba6"). InnerVolumeSpecName "kube-api-access-8ms2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:48:36.023912 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.023892    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ms2m\" (UniqueName: \"kubernetes.io/projected/ba3fb184-2a54-4b53-8591-bf306fd0bba6-kube-api-access-8ms2m\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:36.576402 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.576367    2574 generic.go:356] "Generic (PLEG): container finished" podID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" containerID="9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc" exitCode=0
Jul 25 00:48:36.576879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.576430    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xpt7v"
Jul 25 00:48:36.576879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.576455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xpt7v" event={"ID":"ba3fb184-2a54-4b53-8591-bf306fd0bba6","Type":"ContainerDied","Data":"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"}
Jul 25 00:48:36.576879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.576497    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xpt7v" event={"ID":"ba3fb184-2a54-4b53-8591-bf306fd0bba6","Type":"ContainerDied","Data":"2ec62b35fe533d55d694d29f8bea818bc1540f5a888e3d0ba06683e20dda385d"}
Jul 25 00:48:36.576879 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.576532    2574 scope.go:117] "RemoveContainer" containerID="9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"
Jul 25 00:48:36.585973 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.585944    2574 scope.go:117] "RemoveContainer" containerID="9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"
Jul 25 00:48:36.586219 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:36.586197    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc\": container with ID starting with 9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc not found: ID does not exist" containerID="9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"
Jul 25 00:48:36.586281 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.586231    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc"} err="failed to get container status \"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc\": rpc error: code = NotFound desc = could not find container \"9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc\": container with ID starting with 9ae6207d5543355e0440c4c04512d6209606dc51ce38c4dc69f45f13855bc3bc not found: ID does not exist"
Jul 25 00:48:36.592429 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.592402    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:36.598815 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:36.598794    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xpt7v"]
Jul 25 00:48:37.933162 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:37.933127    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" path="/var/lib/kubelet/pods/ba3fb184-2a54-4b53-8591-bf306fd0bba6/volumes"
Jul 25 00:48:44.721884 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:44.721851    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:44.722292 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:44.722043    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" podUID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" containerName="limitador" containerID="cri-o://50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6" gracePeriod=30
Jul 25 00:48:44.722777 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:44.722758    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:45.272350 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.272326    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:45.403537 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.403427    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6gt4\" (UniqueName: \"kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4\") pod \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") "
Jul 25 00:48:45.403696 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.403561    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file\") pod \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\" (UID: \"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd\") "
Jul 25 00:48:45.403938 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.403907    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file" (OuterVolumeSpecName: "config-file") pod "a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" (UID: "a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:48:45.405522 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.405487    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4" (OuterVolumeSpecName: "kube-api-access-s6gt4") pod "a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" (UID: "a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd"). InnerVolumeSpecName "kube-api-access-s6gt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:48:45.504958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.504933    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-config-file\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:45.504958 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.504957    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gt4\" (UniqueName: \"kubernetes.io/projected/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd-kube-api-access-s6gt4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:45.636302 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.636268    2574 generic.go:356] "Generic (PLEG): container finished" podID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" containerID="50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6" exitCode=0
Jul 25 00:48:45.636476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.636309    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"
Jul 25 00:48:45.636476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.636354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" event={"ID":"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd","Type":"ContainerDied","Data":"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"}
Jul 25 00:48:45.636476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.636405    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z8gmw" event={"ID":"a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd","Type":"ContainerDied","Data":"ad007e22c8f8e5959ec41a7ee82ff8325427f1c23da30445fdcb5471c2f561b2"}
Jul 25 00:48:45.636476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.636428    2574 scope.go:117] "RemoveContainer" containerID="50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"
Jul 25 00:48:45.645329 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.645307    2574 scope.go:117] "RemoveContainer" containerID="50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"
Jul 25 00:48:45.645609 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:45.645590    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6\": container with ID starting with 50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6 not found: ID does not exist" containerID="50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"
Jul 25 00:48:45.645674 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.645616    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6"} err="failed to get container status \"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6\": rpc error: code = NotFound desc = could not find container \"50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6\": container with ID starting with 50a00bbbd46e46461f2e434a1a102337ee4999d06a261a9868785263edc7d7d6 not found: ID does not exist"
Jul 25 00:48:45.657811 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.657759    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:45.660138 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.660119    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z8gmw"]
Jul 25 00:48:45.934380 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:45.934318    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" path="/var/lib/kubelet/pods/a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd/volumes"
Jul 25 00:48:46.595643 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:46.595603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:46.757119 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:46.757079    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:47.111421 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.111387    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"]
Jul 25 00:48:47.111981 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.111959    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" containerName="authorino"
Jul 25 00:48:47.112046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.111983    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" containerName="authorino"
Jul 25 00:48:47.112046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.111996    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" containerName="limitador"
Jul 25 00:48:47.112046 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.112002    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" containerName="limitador"
Jul 25 00:48:47.112145 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.112071    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52058f8-0dd4-4bf1-8c6b-cbd4da4d8fdd" containerName="limitador"
Jul 25 00:48:47.112145 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.112085    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3fb184-2a54-4b53-8591-bf306fd0bba6" containerName="authorino"
Jul 25 00:48:47.116701 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.116683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.120669 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.120250    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 25 00:48:47.120669 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.120309    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 00:48:47.120669 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.120254    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 25 00:48:47.120669 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.120554    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-nbbls\""
Jul 25 00:48:47.123128 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.123106    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"]
Jul 25 00:48:47.141244 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.141210    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:47.217603 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.217579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b34ca81-b42e-4843-8c6a-a760d629edaa-data\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.217721 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.217615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ksn\" (UniqueName: \"kubernetes.io/projected/2b34ca81-b42e-4843-8c6a-a760d629edaa-kube-api-access-n7ksn\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.318250 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.318228    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b34ca81-b42e-4843-8c6a-a760d629edaa-data\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.318366 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.318263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ksn\" (UniqueName: \"kubernetes.io/projected/2b34ca81-b42e-4843-8c6a-a760d629edaa-kube-api-access-n7ksn\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.318626 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.318609    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b34ca81-b42e-4843-8c6a-a760d629edaa-data\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.331911 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.331894    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ksn\" (UniqueName: \"kubernetes.io/projected/2b34ca81-b42e-4843-8c6a-a760d629edaa-kube-api-access-n7ksn\") pod \"postgres-69fb4594cf-8x5bp\" (UID: \"2b34ca81-b42e-4843-8c6a-a760d629edaa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.429132 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.429072    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:47.559254 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.559230    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"]
Jul 25 00:48:47.560208 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:48:47.560186    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b34ca81_b42e_4843_8c6a_a760d629edaa.slice/crio-83cfbed85f5623669e62d8f152bcf38f24047f6a712cdde4f370f8152824e3cf WatchSource:0}: Error finding container 83cfbed85f5623669e62d8f152bcf38f24047f6a712cdde4f370f8152824e3cf: Status 404 returned error can't find the container with id 83cfbed85f5623669e62d8f152bcf38f24047f6a712cdde4f370f8152824e3cf
Jul 25 00:48:47.650000 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:47.649973    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp" event={"ID":"2b34ca81-b42e-4843-8c6a-a760d629edaa","Type":"ContainerStarted","Data":"83cfbed85f5623669e62d8f152bcf38f24047f6a712cdde4f370f8152824e3cf"}
Jul 25 00:48:52.685821 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:52.685785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp" event={"ID":"2b34ca81-b42e-4843-8c6a-a760d629edaa","Type":"ContainerStarted","Data":"22d0ab0e86a3de41670b4fce535a643e4d6685e307b4b663105f593f0f7acf3b"}
Jul 25 00:48:52.686239 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:52.685846    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:52.702522 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:52.702460    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp" podStartSLOduration=0.806760647 podStartE2EDuration="5.702446706s" totalImagesPullingTime="4.895686059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:48:52.700113464 +0000 UTC m=+787.411531863" watchObservedRunningTime="2026-07-25 00:48:52.702446706 +0000 UTC m=+787.413865125"
Jul 25 00:48:58.719793 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:58.719758    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8x5bp"
Jul 25 00:48:59.037867 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.037778    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:59.137932 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.137898    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:48:59.269028 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.268997    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fb8jd"]
Jul 25 00:48:59.275965 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.275941    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fb8jd"]
Jul 25 00:48:59.276302 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.276041    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.278484 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.278459    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nn7tv\""
Jul 25 00:48:59.429255 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.429187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4pcn\" (UniqueName: \"kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn\") pod \"authorino-5d4b67c446-fb8jd\" (UID: \"545f91c1-b70b-476b-bceb-151e617677f7\") " pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.516037 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.516000    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fb8jd"]
Jul 25 00:48:59.516310 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:59.516284    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-w4pcn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-fb8jd" podUID="545f91c1-b70b-476b-bceb-151e617677f7"
Jul 25 00:48:59.530646 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.530612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4pcn\" (UniqueName: \"kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn\") pod \"authorino-5d4b67c446-fb8jd\" (UID: \"545f91c1-b70b-476b-bceb-151e617677f7\") " pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.537686 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.537663    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66585d4cc-rv75j"]
Jul 25 00:48:59.540272 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.540247    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4pcn\" (UniqueName: \"kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn\") pod \"authorino-5d4b67c446-fb8jd\" (UID: \"545f91c1-b70b-476b-bceb-151e617677f7\") " pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.542187 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.542170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.547154 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.547131    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66585d4cc-rv75j"]
Jul 25 00:48:59.628337 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.628305    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66585d4cc-rv75j"]
Jul 25 00:48:59.628652 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:48:59.628630    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hl5zn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-66585d4cc-rv75j" podUID="a187f0b1-ea88-4b2d-9312-f09acdee653b"
Jul 25 00:48:59.631828 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.631805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl5zn\" (UniqueName: \"kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn\") pod \"authorino-66585d4cc-rv75j\" (UID: \"a187f0b1-ea88-4b2d-9312-f09acdee653b\") " pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.653102 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.653072    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:48:59.657291 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.657274    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.659671 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.659651    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 25 00:48:59.663769 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.663746    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:48:59.725212 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.725185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.725620 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.725197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.732253 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.732232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.732400 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.732288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl5zn\" (UniqueName: \"kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn\") pod \"authorino-66585d4cc-rv75j\" (UID: \"a187f0b1-ea88-4b2d-9312-f09acdee653b\") " pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.732477 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.732425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw6zv\" (UniqueName: \"kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.732810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.732787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.736153 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.736136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:48:59.742558 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.742531    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl5zn\" (UniqueName: \"kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn\") pod \"authorino-66585d4cc-rv75j\" (UID: \"a187f0b1-ea88-4b2d-9312-f09acdee653b\") " pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:48:59.833839 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.833808    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4pcn\" (UniqueName: \"kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn\") pod \"545f91c1-b70b-476b-bceb-151e617677f7\" (UID: \"545f91c1-b70b-476b-bceb-151e617677f7\") "
Jul 25 00:48:59.834049 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.833900    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl5zn\" (UniqueName: \"kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn\") pod \"a187f0b1-ea88-4b2d-9312-f09acdee653b\" (UID: \"a187f0b1-ea88-4b2d-9312-f09acdee653b\") "
Jul 25 00:48:59.834049 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.834023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw6zv\" (UniqueName: \"kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.834049 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.834043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.835945 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.835915    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn" (OuterVolumeSpecName: "kube-api-access-w4pcn") pod "545f91c1-b70b-476b-bceb-151e617677f7" (UID: "545f91c1-b70b-476b-bceb-151e617677f7"). InnerVolumeSpecName "kube-api-access-w4pcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:48:59.836051 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.836007    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn" (OuterVolumeSpecName: "kube-api-access-hl5zn") pod "a187f0b1-ea88-4b2d-9312-f09acdee653b" (UID: "a187f0b1-ea88-4b2d-9312-f09acdee653b"). InnerVolumeSpecName "kube-api-access-hl5zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:48:59.836374 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.836355    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.841164 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.841140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw6zv\" (UniqueName: \"kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv\") pod \"authorino-b4559b469-r648f\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") " pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:48:59.934810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.934784    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl5zn\" (UniqueName: \"kubernetes.io/projected/a187f0b1-ea88-4b2d-9312-f09acdee653b-kube-api-access-hl5zn\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:59.934810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.934807    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4pcn\" (UniqueName: \"kubernetes.io/projected/545f91c1-b70b-476b-bceb-151e617677f7-kube-api-access-w4pcn\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:48:59.967862 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:48:59.967837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:49:00.096937 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.096909    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:49:00.098432 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:49:00.098409    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c053b12_5de5_4c59_8b80_7212f869d67c.slice/crio-d73a09fe1dd738b76c9f8cd193b9526aa3c50fdbbf462838f281608429613672 WatchSource:0}: Error finding container d73a09fe1dd738b76c9f8cd193b9526aa3c50fdbbf462838f281608429613672: Status 404 returned error can't find the container with id d73a09fe1dd738b76c9f8cd193b9526aa3c50fdbbf462838f281608429613672
Jul 25 00:49:00.732743 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.732699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4559b469-r648f" event={"ID":"1c053b12-5de5-4c59-8b80-7212f869d67c","Type":"ContainerStarted","Data":"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"}
Jul 25 00:49:00.732743 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.732725    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fb8jd"
Jul 25 00:49:00.732743 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.732740    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66585d4cc-rv75j"
Jul 25 00:49:00.733287 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.732754    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4559b469-r648f" event={"ID":"1c053b12-5de5-4c59-8b80-7212f869d67c","Type":"ContainerStarted","Data":"d73a09fe1dd738b76c9f8cd193b9526aa3c50fdbbf462838f281608429613672"}
Jul 25 00:49:00.751285 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.751234    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b4559b469-r648f" podStartSLOduration=1.4601844179999999 podStartE2EDuration="1.751221344s" totalImagesPullingTime="291.036926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:49:00.748431173 +0000 UTC m=+795.459849572" watchObservedRunningTime="2026-07-25 00:49:00.751221344 +0000 UTC m=+795.462639742"
Jul 25 00:49:00.770050 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.770026    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fb8jd"]
Jul 25 00:49:00.777264 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.777241    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fb8jd"]
Jul 25 00:49:00.796271 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.796247    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66585d4cc-rv75j"]
Jul 25 00:49:00.800555 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:00.800501    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66585d4cc-rv75j"]
Jul 25 00:49:01.933748 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:01.933707    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="545f91c1-b70b-476b-bceb-151e617677f7" path="/var/lib/kubelet/pods/545f91c1-b70b-476b-bceb-151e617677f7/volumes"
Jul 25 00:49:01.934229 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:01.934027    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a187f0b1-ea88-4b2d-9312-f09acdee653b" path="/var/lib/kubelet/pods/a187f0b1-ea88-4b2d-9312-f09acdee653b/volumes"
Jul 25 00:49:05.553387 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:05.553352    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:05.647710 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:05.647669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:05.719775 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:05.719737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:08.936671 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:08.936581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:09.032915 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:09.032881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:15.079960 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:15.079908    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:40.359642 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:40.359597    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:40.457877 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:40.457842    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:40.941835 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:40.941802    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:41.428899 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.428864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:41.529906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.529859    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:41.562712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.562676    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wmwsl"]
Jul 25 00:49:41.567009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.566988    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.569611 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.569592    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fp5wk\""
Jul 25 00:49:41.569734 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.569592    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 25 00:49:41.575947 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.575922    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wmwsl"]
Jul 25 00:49:41.690014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.689936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97665a0e-e7d9-4062-9152-6d17133eb669-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.690014 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.689976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dncm4\" (UniqueName: \"kubernetes.io/projected/97665a0e-e7d9-4062-9152-6d17133eb669-kube-api-access-dncm4\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.791368 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.791328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97665a0e-e7d9-4062-9152-6d17133eb669-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.791559 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.791374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dncm4\" (UniqueName: \"kubernetes.io/projected/97665a0e-e7d9-4062-9152-6d17133eb669-kube-api-access-dncm4\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.792042 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.792022    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97665a0e-e7d9-4062-9152-6d17133eb669-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.799040 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.799018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dncm4\" (UniqueName: \"kubernetes.io/projected/97665a0e-e7d9-4062-9152-6d17133eb669-kube-api-access-dncm4\") pod \"payload-processing-5cc6cd444b-wmwsl\" (UID: \"97665a0e-e7d9-4062-9152-6d17133eb669\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:41.879164 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:41.879129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:42.000270 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:42.000235    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:42.020984 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:42.020957    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wmwsl"]
Jul 25 00:49:42.022029 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:49:42.022006    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97665a0e_e7d9_4062_9152_6d17133eb669.slice/crio-2eebc82e3374c981e87f789de46651c93061542ba3852715d3b0634eb3f7358c WatchSource:0}: Error finding container 2eebc82e3374c981e87f789de46651c93061542ba3852715d3b0634eb3f7358c: Status 404 returned error can't find the container with id 2eebc82e3374c981e87f789de46651c93061542ba3852715d3b0634eb3f7358c
Jul 25 00:49:42.989006 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:42.988971    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl" event={"ID":"97665a0e-e7d9-4062-9152-6d17133eb669","Type":"ContainerStarted","Data":"2eebc82e3374c981e87f789de46651c93061542ba3852715d3b0634eb3f7358c"}
Jul 25 00:49:43.995604 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:43.995565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl" event={"ID":"97665a0e-e7d9-4062-9152-6d17133eb669","Type":"ContainerStarted","Data":"8e2634734efe88b9b562d980bc14bbad86ed79e2df2d7f1d4467e775fa7d762d"}
Jul 25 00:49:43.996068 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:43.995657    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:49:44.009997 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:44.009948    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl" podStartSLOduration=1.511671467 podStartE2EDuration="3.009931045s" totalImagesPullingTime="1.498259578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:49:44.009619855 +0000 UTC m=+838.721038263" watchObservedRunningTime="2026-07-25 00:49:44.009931045 +0000 UTC m=+838.721349447"
Jul 25 00:49:54.665352 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:54.665305    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:54.765667 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:54.765628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:49:55.003715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:49:55.003687    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wmwsl"
Jul 25 00:50:04.168913 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:04.168882    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:04.218739 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:04.218696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:12.173722 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.173689    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"]
Jul 25 00:50:12.178123 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.178104    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.181914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.181882    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 25 00:50:12.182336 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.182275    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 25 00:50:12.182551 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.182531    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gjm4r\""
Jul 25 00:50:12.182755 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.182739    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 25 00:50:12.186009 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.185986    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"]
Jul 25 00:50:12.206032 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.205995    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:12.271476 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.271598 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.271598 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.271676 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271660    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.271712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.271746 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.271731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kncll\" (UniqueName: \"kubernetes.io/projected/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-kube-api-access-kncll\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373077 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373232 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373130    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373232 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373232 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncll\" (UniqueName: \"kubernetes.io/projected/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-kube-api-access-kncll\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373349 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373614 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373591    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.373692 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.373659    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.375351 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.375328    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.375634 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.375616    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.380320 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.380298    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncll\" (UniqueName: \"kubernetes.io/projected/3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5-kube-api-access-kncll\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz\" (UID: \"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.491694 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.491660    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:12.625760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:12.625733    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"]
Jul 25 00:50:12.627063 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:50:12.627038    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3a0dfc_50bf_4ef2_a11e_6e56d20b12a5.slice/crio-b2d48ad8487ece79a77d26bc9a41f963e553f9f30ce531886c10caebcb1a77cc WatchSource:0}: Error finding container b2d48ad8487ece79a77d26bc9a41f963e553f9f30ce531886c10caebcb1a77cc: Status 404 returned error can't find the container with id b2d48ad8487ece79a77d26bc9a41f963e553f9f30ce531886c10caebcb1a77cc
Jul 25 00:50:13.168814 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:13.168773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz" event={"ID":"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5","Type":"ContainerStarted","Data":"b2d48ad8487ece79a77d26bc9a41f963e553f9f30ce531886c10caebcb1a77cc"}
Jul 25 00:50:14.178174 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:14.178086    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz" event={"ID":"3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5","Type":"ContainerStarted","Data":"13f01aa3f844bcaded3753822211fb2fa3fc918b16e040d63aa4489545740b30"}
Jul 25 00:50:14.193352 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:14.193303    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz" podStartSLOduration=0.98153077 podStartE2EDuration="2.193285019s" totalImagesPullingTime="1.211754249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:50:14.19253599 +0000 UTC m=+868.903954406" watchObservedRunningTime="2026-07-25 00:50:14.193285019 +0000 UTC m=+868.904703419"
Jul 25 00:50:22.492268 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:22.492233    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:22.492747 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:22.492298    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:22.504385 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:22.504364    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:23.117293 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:23.117264    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:23.239618 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:23.239590    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz"
Jul 25 00:50:31.707331 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:31.707293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:41.811785 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:41.811751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:50:45.820400 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:45.820373    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:50:45.820882 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:45.820798    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:50:58.123339 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:50:58.123304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:51:18.050731 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:51:18.050693    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:51:18.100238 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:51:18.100211    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:51:23.820231 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:51:23.820187    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:52:22.057933 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.057897    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56bf5dc956-vp2kb"]
Jul 25 00:52:22.061634 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.061613    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.070848 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.070819    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56bf5dc956-vp2kb"]
Jul 25 00:52:22.103771 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.103728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzsvm\" (UniqueName: \"kubernetes.io/projected/bf2e7487-a852-4ee8-9c7c-f539023be05f-kube-api-access-zzsvm\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.103954 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.103786    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf2e7487-a852-4ee8-9c7c-f539023be05f-tls-cert\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.204610 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.204569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzsvm\" (UniqueName: \"kubernetes.io/projected/bf2e7487-a852-4ee8-9c7c-f539023be05f-kube-api-access-zzsvm\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.204805 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.204734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf2e7487-a852-4ee8-9c7c-f539023be05f-tls-cert\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.208042 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.208009    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf2e7487-a852-4ee8-9c7c-f539023be05f-tls-cert\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.212760 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.212730    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzsvm\" (UniqueName: \"kubernetes.io/projected/bf2e7487-a852-4ee8-9c7c-f539023be05f-kube-api-access-zzsvm\") pod \"authorino-56bf5dc956-vp2kb\" (UID: \"bf2e7487-a852-4ee8-9c7c-f539023be05f\") " pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.372303 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.372214    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56bf5dc956-vp2kb"
Jul 25 00:52:22.455042 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.454985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:52:22.519639 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.519610    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56bf5dc956-vp2kb"]
Jul 25 00:52:22.520761 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.520727    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:52:22.521376 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:52:22.521354    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf2e7487_a852_4ee8_9c7c_f539023be05f.slice/crio-d086e0ad185c33e236ea8e65f65841976313b0865a8ced0880b67d7e13c6f566 WatchSource:0}: Error finding container d086e0ad185c33e236ea8e65f65841976313b0865a8ced0880b67d7e13c6f566: Status 404 returned error can't find the container with id d086e0ad185c33e236ea8e65f65841976313b0865a8ced0880b67d7e13c6f566
Jul 25 00:52:22.522741 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.522724    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:52:22.952933 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:22.952898    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56bf5dc956-vp2kb" event={"ID":"bf2e7487-a852-4ee8-9c7c-f539023be05f","Type":"ContainerStarted","Data":"d086e0ad185c33e236ea8e65f65841976313b0865a8ced0880b67d7e13c6f566"}
Jul 25 00:52:23.706435 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:23.706373    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:52:23.786652 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:23.786612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:52:23.963906 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:23.963809    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56bf5dc956-vp2kb" event={"ID":"bf2e7487-a852-4ee8-9c7c-f539023be05f","Type":"ContainerStarted","Data":"912e539923f80a3d67daa7ff6275a510f060d8509b9321cafe545bb2f1eda99c"}
Jul 25 00:52:23.982558 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:23.982472    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56bf5dc956-vp2kb" podStartSLOduration=1.5763965340000001 podStartE2EDuration="1.982447099s" totalImagesPullingTime="406.050565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:52:23.978163809 +0000 UTC m=+998.689582234" watchObservedRunningTime="2026-07-25 00:52:23.982447099 +0000 UTC m=+998.693865499"
Jul 25 00:52:24.005317 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.005274    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:52:24.005575 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.005538    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b4559b469-r648f" podUID="1c053b12-5de5-4c59-8b80-7212f869d67c" containerName="authorino" containerID="cri-o://10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf" gracePeriod=30
Jul 25 00:52:24.277069 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.277034    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:52:24.327585 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.327539    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert\") pod \"1c053b12-5de5-4c59-8b80-7212f869d67c\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") "
Jul 25 00:52:24.327585 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.327592    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw6zv\" (UniqueName: \"kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv\") pod \"1c053b12-5de5-4c59-8b80-7212f869d67c\" (UID: \"1c053b12-5de5-4c59-8b80-7212f869d67c\") "
Jul 25 00:52:24.329797 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.329760    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv" (OuterVolumeSpecName: "kube-api-access-sw6zv") pod "1c053b12-5de5-4c59-8b80-7212f869d67c" (UID: "1c053b12-5de5-4c59-8b80-7212f869d67c"). InnerVolumeSpecName "kube-api-access-sw6zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:52:24.341699 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.341660    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1c053b12-5de5-4c59-8b80-7212f869d67c" (UID: "1c053b12-5de5-4c59-8b80-7212f869d67c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:52:24.428712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.428671    2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c053b12-5de5-4c59-8b80-7212f869d67c-tls-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:52:24.428712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.428708    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw6zv\" (UniqueName: \"kubernetes.io/projected/1c053b12-5de5-4c59-8b80-7212f869d67c-kube-api-access-sw6zv\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:52:24.972403 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.972352    2574 generic.go:356] "Generic (PLEG): container finished" podID="1c053b12-5de5-4c59-8b80-7212f869d67c" containerID="10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf" exitCode=0
Jul 25 00:52:24.972887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.972418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4559b469-r648f" event={"ID":"1c053b12-5de5-4c59-8b80-7212f869d67c","Type":"ContainerDied","Data":"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"}
Jul 25 00:52:24.972887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.972474    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b4559b469-r648f"
Jul 25 00:52:24.972887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.972486    2574 scope.go:117] "RemoveContainer" containerID="10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"
Jul 25 00:52:24.972887 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.972473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b4559b469-r648f" event={"ID":"1c053b12-5de5-4c59-8b80-7212f869d67c","Type":"ContainerDied","Data":"d73a09fe1dd738b76c9f8cd193b9526aa3c50fdbbf462838f281608429613672"}
Jul 25 00:52:24.983389 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.983371    2574 scope.go:117] "RemoveContainer" containerID="10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"
Jul 25 00:52:24.983828 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:52:24.983791    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf\": container with ID starting with 10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf not found: ID does not exist" containerID="10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"
Jul 25 00:52:24.983914 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.983847    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf"} err="failed to get container status \"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf\": rpc error: code = NotFound desc = could not find container \"10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf\": container with ID starting with 10f5b270e3bd6cc4d14d87b9820d1efbc1e9c8860867fe56638b8aa496d342bf not found: ID does not exist"
Jul 25 00:52:24.994310 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.994283    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:52:24.996807 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:24.996787    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b4559b469-r648f"]
Jul 25 00:52:25.934240 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:52:25.934207    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c053b12-5de5-4c59-8b80-7212f869d67c" path="/var/lib/kubelet/pods/1c053b12-5de5-4c59-8b80-7212f869d67c/volumes"
Jul 25 00:53:09.609975 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:53:09.609904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:53:09.710831 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:53:09.710787    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:54:07.200047 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:07.200006    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:54:07.251528 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:07.251452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:54:12.310818 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.310784    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:12.311256 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.311239    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c053b12-5de5-4c59-8b80-7212f869d67c" containerName="authorino"
Jul 25 00:54:12.311302 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.311260    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c053b12-5de5-4c59-8b80-7212f869d67c" containerName="authorino"
Jul 25 00:54:12.311365 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.311355    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c053b12-5de5-4c59-8b80-7212f869d67c" containerName="authorino"
Jul 25 00:54:12.314616 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.314600    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.317846 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.317824    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 00:54:12.317967 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.317883    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 00:54:12.317967 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.317884    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-f5vw4\""
Jul 25 00:54:12.324715 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.324695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:12.489919 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.489883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9hs\" (UniqueName: \"kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.490109 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.490004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.591273 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.591182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.591426 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.591338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9hs\" (UniqueName: \"kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.593851 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.593828    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.605589 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.600110    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9hs\" (UniqueName: \"kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs\") pod \"maas-controller-6cfc5bd855-4xsgg\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") " pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.626964 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.626937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:12.771104 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:12.771079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:12.772813 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:54:12.772785    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170930df_defb_4f2a_909c_aea9419fd57e.slice/crio-e1864f88e4802143b5890f7612d05baed7317dda797cb17a21d3f317299e6c4c WatchSource:0}: Error finding container e1864f88e4802143b5890f7612d05baed7317dda797cb17a21d3f317299e6c4c: Status 404 returned error can't find the container with id e1864f88e4802143b5890f7612d05baed7317dda797cb17a21d3f317299e6c4c
Jul 25 00:54:13.638493 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:13.638453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" event={"ID":"170930df-defb-4f2a-909c-aea9419fd57e","Type":"ContainerStarted","Data":"e1864f88e4802143b5890f7612d05baed7317dda797cb17a21d3f317299e6c4c"}
Jul 25 00:54:15.652842 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:15.652801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" event={"ID":"170930df-defb-4f2a-909c-aea9419fd57e","Type":"ContainerStarted","Data":"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"}
Jul 25 00:54:15.653255 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:15.653008    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:15.670783 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:15.670732    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" podStartSLOduration=1.311336641 podStartE2EDuration="3.670717092s" totalImagesPullingTime="2.359380451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:54:15.669264137 +0000 UTC m=+1110.380682548" watchObservedRunningTime="2026-07-25 00:54:15.670717092 +0000 UTC m=+1110.382135492"
Jul 25 00:54:26.660807 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:26.660778    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:31.212917 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.212873    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:54:31.325725 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.325690    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:31.326018 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.325983    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" podUID="170930df-defb-4f2a-909c-aea9419fd57e" containerName="manager" containerID="cri-o://2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40" gracePeriod=30
Jul 25 00:54:31.594402 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.594375    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:31.665810 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.665780    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9hs\" (UniqueName: \"kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs\") pod \"170930df-defb-4f2a-909c-aea9419fd57e\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") "
Jul 25 00:54:31.665971 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.665852    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert\") pod \"170930df-defb-4f2a-909c-aea9419fd57e\" (UID: \"170930df-defb-4f2a-909c-aea9419fd57e\") "
Jul 25 00:54:31.667846 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.667820    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "170930df-defb-4f2a-909c-aea9419fd57e" (UID: "170930df-defb-4f2a-909c-aea9419fd57e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:54:31.667944 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.667917    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs" (OuterVolumeSpecName: "kube-api-access-gx9hs") pod "170930df-defb-4f2a-909c-aea9419fd57e" (UID: "170930df-defb-4f2a-909c-aea9419fd57e"). InnerVolumeSpecName "kube-api-access-gx9hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:54:31.752316 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.752240    2574 generic.go:356] "Generic (PLEG): container finished" podID="170930df-defb-4f2a-909c-aea9419fd57e" containerID="2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40" exitCode=0
Jul 25 00:54:31.752316 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.752277    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg"
Jul 25 00:54:31.752443 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.752323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" event={"ID":"170930df-defb-4f2a-909c-aea9419fd57e","Type":"ContainerDied","Data":"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"}
Jul 25 00:54:31.752443 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.752371    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-4xsgg" event={"ID":"170930df-defb-4f2a-909c-aea9419fd57e","Type":"ContainerDied","Data":"e1864f88e4802143b5890f7612d05baed7317dda797cb17a21d3f317299e6c4c"}
Jul 25 00:54:31.752443 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.752396    2574 scope.go:117] "RemoveContainer" containerID="2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"
Jul 25 00:54:31.761820 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.761804    2574 scope.go:117] "RemoveContainer" containerID="2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"
Jul 25 00:54:31.762079 ip-10-0-141-233 kubenswrapper[2574]: E0725 00:54:31.762060    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40\": container with ID starting with 2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40 not found: ID does not exist" containerID="2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"
Jul 25 00:54:31.762129 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.762087    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40"} err="failed to get container status \"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40\": rpc error: code = NotFound desc = could not find container \"2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40\": container with ID starting with 2a8b3c7083f598be7907389fa4076c79f27df8e9036848b0883e3549a3661d40 not found: ID does not exist"
Jul 25 00:54:31.766449 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.766431    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170930df-defb-4f2a-909c-aea9419fd57e-webhook-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:54:31.766547 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.766452    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9hs\" (UniqueName: \"kubernetes.io/projected/170930df-defb-4f2a-909c-aea9419fd57e-kube-api-access-gx9hs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:54:31.773182 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.773162    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:31.776616 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.776596    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-4xsgg"]
Jul 25 00:54:31.933999 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:31.933971    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170930df-defb-4f2a-909c-aea9419fd57e" path="/var/lib/kubelet/pods/170930df-defb-4f2a-909c-aea9419fd57e/volumes"
Jul 25 00:54:32.459977 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.459943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:54:32.566920 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.566885    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-hmd7l"]
Jul 25 00:54:32.567600 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.567582    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170930df-defb-4f2a-909c-aea9419fd57e" containerName="manager"
Jul 25 00:54:32.567653 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.567603    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="170930df-defb-4f2a-909c-aea9419fd57e" containerName="manager"
Jul 25 00:54:32.567728 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.567716    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="170930df-defb-4f2a-909c-aea9419fd57e" containerName="manager"
Jul 25 00:54:32.572603 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.572586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.574751 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.574725    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 00:54:32.575608 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.575590    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-f5vw4\""
Jul 25 00:54:32.575711 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.575631    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 00:54:32.579940 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.579919    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-hmd7l"]
Jul 25 00:54:32.675893 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.675867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzc6v\" (UniqueName: \"kubernetes.io/projected/5eb21b43-334a-4280-ac46-6d098b104cbf-kube-api-access-bzc6v\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.676013 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.675900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eb21b43-334a-4280-ac46-6d098b104cbf-webhook-cert\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.776718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.776652    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzc6v\" (UniqueName: \"kubernetes.io/projected/5eb21b43-334a-4280-ac46-6d098b104cbf-kube-api-access-bzc6v\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.776718 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.776681    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eb21b43-334a-4280-ac46-6d098b104cbf-webhook-cert\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.779158 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.779140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eb21b43-334a-4280-ac46-6d098b104cbf-webhook-cert\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.784616 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.784593    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzc6v\" (UniqueName: \"kubernetes.io/projected/5eb21b43-334a-4280-ac46-6d098b104cbf-kube-api-access-bzc6v\") pod \"maas-controller-6cfc5bd855-hmd7l\" (UID: \"5eb21b43-334a-4280-ac46-6d098b104cbf\") " pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:32.884004 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:32.883980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:33.019532 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:33.019476    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cfc5bd855-hmd7l"]
Jul 25 00:54:33.021127 ip-10-0-141-233 kubenswrapper[2574]: W0725 00:54:33.021102    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb21b43_334a_4280_ac46_6d098b104cbf.slice/crio-fbb4c183cbafad5dd0f1ac395a2c7c09af3b2c235ec70b1869466517ca94b21f WatchSource:0}: Error finding container fbb4c183cbafad5dd0f1ac395a2c7c09af3b2c235ec70b1869466517ca94b21f: Status 404 returned error can't find the container with id fbb4c183cbafad5dd0f1ac395a2c7c09af3b2c235ec70b1869466517ca94b21f
Jul 25 00:54:33.766145 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:33.766113    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l" event={"ID":"5eb21b43-334a-4280-ac46-6d098b104cbf","Type":"ContainerStarted","Data":"20efb1ca636798f5c7a0870e6d93a15fb6e166fcf186cacdce25b5530ac5c58b"}
Jul 25 00:54:33.766578 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:33.766162    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l" event={"ID":"5eb21b43-334a-4280-ac46-6d098b104cbf","Type":"ContainerStarted","Data":"fbb4c183cbafad5dd0f1ac395a2c7c09af3b2c235ec70b1869466517ca94b21f"}
Jul 25 00:54:33.766578 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:33.766251    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:33.792039 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:33.791988    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l" podStartSLOduration=1.429331538 podStartE2EDuration="1.791971479s" totalImagesPullingTime="362.639941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:54:33.780050629 +0000 UTC m=+1128.491469018" watchObservedRunningTime="2026-07-25 00:54:33.791971479 +0000 UTC m=+1128.503389877"
Jul 25 00:54:44.774250 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:44.774219    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cfc5bd855-hmd7l"
Jul 25 00:54:52.604548 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:54:52.604492    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:08.127656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:08.127611    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:08.191138 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:08.191052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:08.360763 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:08.360726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:13.925172 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:13.925137    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:14.026991 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:14.026957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:14.125587 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:14.125553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:14.289663 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:14.289632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:17.565688 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:17.565651    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:19.944657 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:19.944622    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:20.045522 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:20.045472    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:55:45.876070 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:45.876039    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:55:45.880386 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:55:45.880363    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 00:56:22.465023 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:56:22.464985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 00:59:22.311268 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:22.311228    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:59:22.311845 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:22.311452    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" podUID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" containerName="manager" containerID="cri-o://c211e02bf36b7387a00e735358b33edd377a21fd057e987676d9aba24f801dd8" gracePeriod=10
Jul 25 00:59:25.554848 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.554815    2574 generic.go:356] "Generic (PLEG): container finished" podID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" containerID="c211e02bf36b7387a00e735358b33edd377a21fd057e987676d9aba24f801dd8" exitCode=0
Jul 25 00:59:25.555207 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.554871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" event={"ID":"1be15fde-945e-44ba-847f-2cab3ee5b3cb","Type":"ContainerDied","Data":"c211e02bf36b7387a00e735358b33edd377a21fd057e987676d9aba24f801dd8"}
Jul 25 00:59:25.676003 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.675974    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:59:25.763210 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.763181    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn6hs\" (UniqueName: \"kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs\") pod \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") "
Jul 25 00:59:25.763384 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.763265    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume\") pod \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\" (UID: \"1be15fde-945e-44ba-847f-2cab3ee5b3cb\") "
Jul 25 00:59:25.763705 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.763674    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1be15fde-945e-44ba-847f-2cab3ee5b3cb" (UID: "1be15fde-945e-44ba-847f-2cab3ee5b3cb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:59:25.765176 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.765150    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs" (OuterVolumeSpecName: "kube-api-access-mn6hs") pod "1be15fde-945e-44ba-847f-2cab3ee5b3cb" (UID: "1be15fde-945e-44ba-847f-2cab3ee5b3cb"). InnerVolumeSpecName "kube-api-access-mn6hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:59:25.864712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.864683    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1be15fde-945e-44ba-847f-2cab3ee5b3cb-extensions-socket-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:59:25.864712 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:25.864709    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn6hs\" (UniqueName: \"kubernetes.io/projected/1be15fde-945e-44ba-847f-2cab3ee5b3cb-kube-api-access-mn6hs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 00:59:26.562577 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:26.562539    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj" event={"ID":"1be15fde-945e-44ba-847f-2cab3ee5b3cb","Type":"ContainerDied","Data":"561ed35a41d213858f84b5954c36fae8031c6430a4aa1f0a7b085a4d472632c8"}
Jul 25 00:59:26.562577 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:26.562561    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"
Jul 25 00:59:26.563077 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:26.562610    2574 scope.go:117] "RemoveContainer" containerID="c211e02bf36b7387a00e735358b33edd377a21fd057e987676d9aba24f801dd8"
Jul 25 00:59:26.581106 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:26.581080    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:59:26.584656 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:26.584632    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zp6sj"]
Jul 25 00:59:27.933998 ip-10-0-141-233 kubenswrapper[2574]: I0725 00:59:27.933967    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" path="/var/lib/kubelet/pods/1be15fde-945e-44ba-847f-2cab3ee5b3cb/volumes"
Jul 25 01:00:00.141541 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.141486    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"]
Jul 25 01:00:00.142176 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.142155    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" containerName="manager"
Jul 25 01:00:00.142265 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.142180    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" containerName="manager"
Jul 25 01:00:00.142363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.142347    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be15fde-945e-44ba-847f-2cab3ee5b3cb" containerName="manager"
Jul 25 01:00:00.146747 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.146725    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"
Jul 25 01:00:00.149142 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.149119    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:00:00.150841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.150814    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"]
Jul 25 01:00:00.159894 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.159875    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"
Jul 25 01:00:00.499859 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.499830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"]
Jul 25 01:00:00.501070 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:00:00.501050    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed089de_2db1_4a45_ba43_809e0f31037e.slice/crio-75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633 WatchSource:0}: Error finding container 75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633: Status 404 returned error can't find the container with id 75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633
Jul 25 01:00:00.503390 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.503376    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:00:00.765466 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:00.765377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz" event={"ID":"4ed089de-2db1-4a45-ba43-809e0f31037e","Type":"ContainerStarted","Data":"75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633"}
Jul 25 01:00:02.780115 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:02.780076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz" event={"ID":"4ed089de-2db1-4a45-ba43-809e0f31037e","Type":"ContainerStarted","Data":"5bf8e940aafe6cd50df80c32d82ea3b07e3f608b3c9bf7e2be590eed6546c396"}
Jul 25 01:00:03.788165 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:03.788129    2574 generic.go:356] "Generic (PLEG): container finished" podID="4ed089de-2db1-4a45-ba43-809e0f31037e" containerID="5bf8e940aafe6cd50df80c32d82ea3b07e3f608b3c9bf7e2be590eed6546c396" exitCode=0
Jul 25 01:00:03.788603 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:03.788228    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz" event={"ID":"4ed089de-2db1-4a45-ba43-809e0f31037e","Type":"ContainerDied","Data":"5bf8e940aafe6cd50df80c32d82ea3b07e3f608b3c9bf7e2be590eed6546c396"}
Jul 25 01:00:04.942178 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:04.942145    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"
Jul 25 01:00:05.801058 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:05.801028    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz"
Jul 25 01:00:05.801316 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:05.801065    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749020-mjlkz" event={"ID":"4ed089de-2db1-4a45-ba43-809e0f31037e","Type":"ContainerDied","Data":"75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633"}
Jul 25 01:00:05.801316 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:05.801139    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75643302fd1e3eaeb790d7adb2a07c101fb4793f40ef9b9b1a55d731a3a96633"
Jul 25 01:00:08.185182 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:08.185142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:00:23.472164 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.472124    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"]
Jul 25 01:00:23.472609 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.472595    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed089de-2db1-4a45-ba43-809e0f31037e" containerName="cleanup"
Jul 25 01:00:23.472666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.472611    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed089de-2db1-4a45-ba43-809e0f31037e" containerName="cleanup"
Jul 25 01:00:23.472702 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.472689    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed089de-2db1-4a45-ba43-809e0f31037e" containerName="cleanup"
Jul 25 01:00:23.475883 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.475867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.478152 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.478126    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qvbjg\""
Jul 25 01:00:23.586712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.586680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.586873 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.586789    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjn2\" (UniqueName: \"kubernetes.io/projected/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-kube-api-access-rkjn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.687313 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.687282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.687462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.687369    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjn2\" (UniqueName: \"kubernetes.io/projected/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-kube-api-access-rkjn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.687685 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.687666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.695638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.695614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjn2\" (UniqueName: \"kubernetes.io/projected/61feac24-fe2a-44e7-a4fd-d6a21c347d9d-kube-api-access-rkjn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qz8rl\" (UID: \"61feac24-fe2a-44e7-a4fd-d6a21c347d9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:23.786357 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:23.786264    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:26.377349 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:26.377311    2574 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 25 01:00:26.377349 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83" Netns:"/var/run/netns/6a5f10db-9ef5-4e55-9536-259cd95e98c8" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:26.377349 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:26.377349 ip-10-0-141-233 kubenswrapper[2574]:  >
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:26.377371    2574 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83" Netns:"/var/run/netns/6a5f10db-9ef5-4e55-9536-259cd95e98c8" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:26.377392    2574 kuberuntime_manager.go:1353] "CreatePodSandbox for pod failed" err=<
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83" Netns:"/var/run/netns/6a5f10db-9ef5-4e55-9536-259cd95e98c8" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:26.377976 ip-10-0-141-233 kubenswrapper[2574]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:26.378419 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:26.377456    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system(61feac24-fe2a-44e7-a4fd-d6a21c347d9d)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system(61feac24-fe2a-44e7-a4fd-d6a21c347d9d)\\\": rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network \\\"multus-cni-network\\\": plugin type=\\\"multus-shim\\\" name=\\\"multus-cni-network\\\" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:\\\"ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83\\\" Netns:\\\"/var/run/netns/6a5f10db-9ef5-4e55-9536-259cd95e98c8\\\" IfName:\\\"eth0\\\" Args:\\\"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=ff5611bf9508c97b85514e5c8d7f0e19105e9667d6299097b5fae46b37302b83;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d\\\" Path:\\\"\\\" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get \\\"https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s\\\": context deadline exceeded\\n': StdinData: {\\\"auxiliaryCNIChainName\\\":\\\"vendor-cni-chain\\\",\\\"binDir\\\":\\\"/var/lib/cni/bin\\\",\\\"clusterNetwork\\\":\\\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\\\",\\\"cniVersion\\\":\\\"0.3.1\\\",\\\"daemonSocketDir\\\":\\\"/run/multus/socket\\\",\\\"globalNamespaces\\\":\\\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\\\",\\\"logLevel\\\":\\\"verbose\\\",\\\"logToStderr\\\":true,\\\"name\\\":\\\"multus-cni-network\\\",\\\"namespaceIsolation\\\":true,\\\"type\\\":\\\"multus-shim\\\"}\"" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl" podUID="61feac24-fe2a-44e7-a4fd-d6a21c347d9d"
Jul 25 01:00:40.926589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:40.926560    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:40.927147 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:40.926787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:43.519770 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:43.519737    2574 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 25 01:00:43.519770 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2" Netns:"/var/run/netns/28373106-376c-4c22-a179-11b581f41c30" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:43.519770 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:43.519770 ip-10-0-141-233 kubenswrapper[2574]:  >
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:43.519789    2574 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2" Netns:"/var/run/netns/28373106-376c-4c22-a179-11b581f41c30" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:43.519806    2574 kuberuntime_manager.go:1353] "CreatePodSandbox for pod failed" err=<
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2" Netns:"/var/run/netns/28373106-376c-4c22-a179-11b581f41c30" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get "https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s": context deadline exceeded
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 25 01:00:43.520275 ip-10-0-141-233 kubenswrapper[2574]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:43.520712 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:00:43.519878    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system(61feac24-fe2a-44e7-a4fd-d6a21c347d9d)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system(61feac24-fe2a-44e7-a4fd-d6a21c347d9d)\\\": rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-qz8rl_kuadrant-system_61feac24-fe2a-44e7-a4fd-d6a21c347d9d_0(a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl to CNI network \\\"multus-cni-network\\\": plugin type=\\\"multus-shim\\\" name=\\\"multus-cni-network\\\" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:\\\"a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2\\\" Netns:\\\"/var/run/netns/28373106-376c-4c22-a179-11b581f41c30\\\" IfName:\\\"eth0\\\" Args:\\\"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-86878c769f-qz8rl;K8S_POD_INFRA_CONTAINER_ID=a49d502031f24bea5e5dbe9385244418153d03abb8013f06d2993fda170acba2;K8S_POD_UID=61feac24-fe2a-44e7-a4fd-d6a21c347d9d\\\" Path:\\\"\\\" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl/61feac24-fe2a-44e7-a4fd-d6a21c347d9d]: error waiting for pod: Get \\\"https://a3a774fc6e4a14210a7052df5311a846-85ee8da77f2aef71.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-86878c769f-qz8rl?timeout=1m0s\\\": context deadline exceeded\\n': StdinData: {\\\"auxiliaryCNIChainName\\\":\\\"vendor-cni-chain\\\",\\\"binDir\\\":\\\"/var/lib/cni/bin\\\",\\\"clusterNetwork\\\":\\\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\\\",\\\"cniVersion\\\":\\\"0.3.1\\\",\\\"daemonSocketDir\\\":\\\"/run/multus/socket\\\",\\\"globalNamespaces\\\":\\\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\\\",\\\"logLevel\\\":\\\"verbose\\\",\\\"logToStderr\\\":true,\\\"name\\\":\\\"multus-cni-network\\\",\\\"namespaceIsolation\\\":true,\\\"type\\\":\\\"multus-shim\\\"}\"" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl" podUID="61feac24-fe2a-44e7-a4fd-d6a21c347d9d"
Jul 25 01:00:45.917416 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:45.917389    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:00:45.924062 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:45.924037    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:00:55.930991 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:55.930961    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:55.931385 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:55.931195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:57.591529 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:57.591474    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"]
Jul 25 01:00:57.743576 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:57.743550    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"]
Jul 25 01:00:57.745013 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:00:57.744984    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61feac24_fe2a_44e7_a4fd_d6a21c347d9d.slice/crio-d49a37a64f22d8c15811264284371363b770c33aad6b8ffff97ffbc7f5afd06d WatchSource:0}: Error finding container d49a37a64f22d8c15811264284371363b770c33aad6b8ffff97ffbc7f5afd06d: Status 404 returned error can't find the container with id d49a37a64f22d8c15811264284371363b770c33aad6b8ffff97ffbc7f5afd06d
Jul 25 01:00:58.116921 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:58.116886    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl" event={"ID":"61feac24-fe2a-44e7-a4fd-d6a21c347d9d","Type":"ContainerStarted","Data":"44bc15836c7056b0223de68f6926ab5cbc2a7cf86bb9a3939ae9056d4014f1fa"}
Jul 25 01:00:58.117099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:58.116937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl" event={"ID":"61feac24-fe2a-44e7-a4fd-d6a21c347d9d","Type":"ContainerStarted","Data":"d49a37a64f22d8c15811264284371363b770c33aad6b8ffff97ffbc7f5afd06d"}
Jul 25 01:00:58.117099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:58.116995    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:00:58.135534 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:00:58.135445    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl" podStartSLOduration=35.135428728 podStartE2EDuration="35.135428728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:00:58.134414183 +0000 UTC m=+1512.845832608" watchObservedRunningTime="2026-07-25 01:00:58.135428728 +0000 UTC m=+1512.846847128"
Jul 25 01:01:04.623585 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:04.623546    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:04.696565 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:04.696484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:04.811174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:04.811142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:04.951285 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:04.951190    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.032107 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.032074    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.109708 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.109673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.180988 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.180953    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.311644 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.311612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.362905 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.362882    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.842889 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.842860    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:05.946049 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:05.946017    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:06.069439 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:06.069405    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:06.169254 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:06.169182    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:06.269393 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:06.269365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:06.373439 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:06.373409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:09.125638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:09.125548    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qz8rl"
Jul 25 01:01:10.686923 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:10.686886    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:10.818019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:10.817985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:10.916605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:10.916571    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:10.995500 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:10.995468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.109128 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.109096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.318215 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.318138    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.395319 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.395287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.495282 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.495253    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.596042 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.595977    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.731821 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.731790    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:11.832424 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:11.832398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:12.151568 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:12.151538    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:12.248736 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:12.248706    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:12.944430 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:12.944392    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:13.044795 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:13.044768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:13.337523 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:13.337474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:13.438738 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:13.438691    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:54.000116 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:54.000079    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:01:55.611888 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:01:55.611849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:02:05.322403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:02:05.322369    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:02:05.371256 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:02:05.371226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:03:41.508022 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:03:41.507985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:04:05.196671 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:04:05.196638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:04:34.423585 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:04:34.423544    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:05:20.337856 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:05:20.337817    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:05:21.987712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:05:21.987643    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:05:45.959997 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:05:45.959964    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:05:45.969420 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:05:45.969396    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:08:36.410771 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:08:36.410739    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:08:57.928709 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:08:57.928629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:01.104560 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:01.104493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:01.202533 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:01.202472    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:01.572379 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:01.572338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:07.245503 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.245467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:07.345943 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.345906    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:07.492438 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.492405    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:07.781164 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.781130    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:07.785235 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.785216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.787606 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.787583    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-82216ebf-openshift-default-dockercfg-k726x\""
Jul 25 01:09:07.805837 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.805815    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:07.806301 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.806275    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:07.936936 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.936903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.936945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.936972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937077    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937115    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:07.937403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:07.937316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pj7q\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pj7q\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038373    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038520    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038554    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038815 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.038971 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.039594 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.038976    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.039759 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.039405    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.039865 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.039541    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.041314 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.041288    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.041568 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.041547    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.046338 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.046312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.046720 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.046701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pj7q\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q\") pod \"e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") " pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.095166 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.095137    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:08.249246 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.249217    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:08.250613 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:09:08.250580    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b689f77_e600_4bc4_974d_c5a8da9471bf.slice/crio-a51aaaab5fffa0fd7eb689b4419857e5b4468246a296dd8059159061d69b74e4 WatchSource:0}: Error finding container a51aaaab5fffa0fd7eb689b4419857e5b4468246a296dd8059159061d69b74e4: Status 404 returned error can't find the container with id a51aaaab5fffa0fd7eb689b4419857e5b4468246a296dd8059159061d69b74e4
Jul 25 01:09:08.252762 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.252729    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:08.252825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.252795    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:08.252860 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.252824    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:08.262572 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:08.262498    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:09.168565 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:09.168531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerStarted","Data":"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"}
Jul 25 01:09:09.168735 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:09.168585    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerStarted","Data":"a51aaaab5fffa0fd7eb689b4419857e5b4468246a296dd8059159061d69b74e4"}
Jul 25 01:09:09.190536 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:09.190467    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podStartSLOduration=2.190453048 podStartE2EDuration="2.190453048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:09:09.186692691 +0000 UTC m=+2003.898111089" watchObservedRunningTime="2026-07-25 01:09:09.190453048 +0000 UTC m=+2003.901871498"
Jul 25 01:09:10.095611 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.095576    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:10.101885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.101858    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:10.175407 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.175383    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:10.176260 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.176242    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:10.640948 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.640911    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:10.730801 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.730768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:10.798697 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:10.798664    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:11.190475 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.190445    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:11.291807 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.291774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:11.297267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.297239    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:11.301003 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.300981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.303298 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.303275    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-82216ebf\""
Jul 25 01:09:11.303410 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.303282    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-82216ebf-dockercfg-bl6pg\""
Jul 25 01:09:11.313092 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.313069    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:11.475524 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.475486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mq2z\" (UniqueName: \"kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.475682 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.475591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.576936 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.576899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.577099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.577021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mq2z\" (UniqueName: \"kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.577682 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.577662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.585174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.585145    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mq2z\" (UniqueName: \"kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z\") pod \"payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.613666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.613639    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:11.977844 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:11.977817    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:11.978448 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:09:11.978420    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21adb48a_2c7c_4f6c_95f2_93b08a4ed1e7.slice/crio-4459df396f7d139ebf972482d6d4cc1966bdcebcd17fea20eadcc102b0e9ee9d WatchSource:0}: Error finding container 4459df396f7d139ebf972482d6d4cc1966bdcebcd17fea20eadcc102b0e9ee9d: Status 404 returned error can't find the container with id 4459df396f7d139ebf972482d6d4cc1966bdcebcd17fea20eadcc102b0e9ee9d
Jul 25 01:09:12.189073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:12.189034    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" event={"ID":"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7","Type":"ContainerStarted","Data":"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"}
Jul 25 01:09:12.189239 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:12.189128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" event={"ID":"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7","Type":"ContainerStarted","Data":"4459df396f7d139ebf972482d6d4cc1966bdcebcd17fea20eadcc102b0e9ee9d"}
Jul 25 01:09:12.189292 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:12.189274    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:12.207928 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:12.207883    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" podStartSLOduration=1.207869874 podStartE2EDuration="1.207869874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:09:12.20592302 +0000 UTC m=+2006.917341419" watchObservedRunningTime="2026-07-25 01:09:12.207869874 +0000 UTC m=+2006.919288273"
Jul 25 01:09:23.197700 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:23.197663    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:25.340078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:25.339526    2574 patch_prober.go:28] interesting pod/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:41148->10.132.0.71:15021: read: connection reset by peer" start-of-body=
Jul 25 01:09:25.340078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:25.339597    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:41148->10.132.0.71:15021: read: connection reset by peer"
Jul 25 01:09:26.299026 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.298991    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerID="a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4" exitCode=0
Jul 25 01:09:26.299204 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.299064    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerDied","Data":"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"}
Jul 25 01:09:26.299494 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.299477    2574 scope.go:117] "RemoveContainer" containerID="a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"
Jul 25 01:09:26.300579 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.300549    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:26.300646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.300635    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:26.300695 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:26.300681    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:09:27.310842 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:27.310807    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerStarted","Data":"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"}
Jul 25 01:09:28.095739 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:28.095704    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:28.095937 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:28.095900    2574 patch_prober.go:28] interesting pod/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 25 01:09:28.096019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:28.095953    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 25 01:09:29.095853 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:29.095816    2574 patch_prober.go:28] interesting pod/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 25 01:09:29.096282 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:29.095868    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 25 01:09:30.095974 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:30.095942    2574 patch_prober.go:28] interesting pod/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 25 01:09:30.096381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:30.095991    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 25 01:09:31.099428 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:31.099397    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:31.099825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:31.099748    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:31.100428 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:31.100411    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:43.716190 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:43.716154    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:46.547287 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.547248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:46.590901 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.590865    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:46.591169 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.591116    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" podUID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" containerName="payload-pre-processing" containerID="cri-o://ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195" gracePeriod=30
Jul 25 01:09:46.613696 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.613667    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:46.670620 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.670590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:46.862297 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:46.862273    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:47.029464 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.029430    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume\") pod \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") "
Jul 25 01:09:47.029659 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.029490    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mq2z\" (UniqueName: \"kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z\") pod \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\" (UID: \"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7\") "
Jul 25 01:09:47.029807 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.029784    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" (UID: "21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:09:47.031470 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.031450    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z" (OuterVolumeSpecName: "kube-api-access-4mq2z") pod "21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" (UID: "21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7"). InnerVolumeSpecName "kube-api-access-4mq2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:09:47.130777 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.130693    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-plugins-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:47.130777 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.130722    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mq2z\" (UniqueName: \"kubernetes.io/projected/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7-kube-api-access-4mq2z\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:47.429912 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.429822    2574 generic.go:356] "Generic (PLEG): container finished" podID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" containerID="ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195" exitCode=0
Jul 25 01:09:47.429912 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.429870    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"
Jul 25 01:09:47.430102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.429912    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" event={"ID":"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7","Type":"ContainerDied","Data":"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"}
Jul 25 01:09:47.430102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.429961    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr" event={"ID":"21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7","Type":"ContainerDied","Data":"4459df396f7d139ebf972482d6d4cc1966bdcebcd17fea20eadcc102b0e9ee9d"}
Jul 25 01:09:47.430102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.429982    2574 scope.go:117] "RemoveContainer" containerID="ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"
Jul 25 01:09:47.442155 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.442138    2574 scope.go:117] "RemoveContainer" containerID="ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"
Jul 25 01:09:47.442457 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:09:47.442439    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195\": container with ID starting with ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195 not found: ID does not exist" containerID="ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"
Jul 25 01:09:47.442582 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.442466    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195"} err="failed to get container status \"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195\": rpc error: code = NotFound desc = could not find container \"ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195\": container with ID starting with ed527bf1e6b36e2691c13e1a0450669ab1b86b9db7452488d55f2223d215f195 not found: ID does not exist"
Jul 25 01:09:47.453495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.453452    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:47.456836 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.456815    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-82216ebf-65787cc988-fcxsr"]
Jul 25 01:09:47.933630 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:47.933596    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" path="/var/lib/kubelet/pods/21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7/volumes"
Jul 25 01:09:51.221203 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.221170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:51.423140 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.423112    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:51.423332 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.423304    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy" containerID="cri-o://7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12" gracePeriod=30
Jul 25 01:09:51.429163 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.429133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:51.659673 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.659581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:51.934668 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:51.934589    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:52.036716 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:52.036686    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:52.162032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:52.162002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:52.509686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:52.509649    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:53.135557 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:53.135520    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:55.596182 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.596147    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:55.955904 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.955878    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:09:55.956626 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.956609    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" containerName="payload-pre-processing"
Jul 25 01:09:55.956679 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.956631    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" containerName="payload-pre-processing"
Jul 25 01:09:55.956775 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.956755    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21adb48a-2c7c-4f6c-95f2-93b08a4ed1e7" containerName="payload-pre-processing"
Jul 25 01:09:55.960618 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.960597    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:55.963008 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.962989    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9j2xk\""
Jul 25 01:09:55.963288 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.963014    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0b6fd8e2-serving-cert\""
Jul 25 01:09:55.968452 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.968428    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:09:55.984144 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:55.984108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:56.009973 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.009947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.010069 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.009977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.010069 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.010016    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt2ph\" (UniqueName: \"kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.110697 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.110668    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.110848 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.110701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.110848 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.110737    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt2ph\" (UniqueName: \"kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.110848 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:09:56.110830    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-serving-cert: secret "maas-api-e2e-migrate-0b6fd8e2-serving-cert" not found
Jul 25 01:09:56.110974 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:09:56.110892    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls podName:d53be8ca-f382-43f4-84f1-40b686a5ca9d nodeName:}" failed. No retries permitted until 2026-07-25 01:09:56.610875816 +0000 UTC m=+2051.322294197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls") pod "maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" (UID: "d53be8ca-f382-43f4-84f1-40b686a5ca9d") : secret "maas-api-e2e-migrate-0b6fd8e2-serving-cert" not found
Jul 25 01:09:56.111461 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.111439    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.119148 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.119122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt2ph\" (UniqueName: \"kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.301325 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.301241    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:56.416241 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.416210    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:56.515280 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.515250    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:09:56.557589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.557526    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:09:56.561135 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.561116    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.563520 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.563488    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0b6fd8e2\""
Jul 25 01:09:56.563626 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.563489    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0b6fd8e2-dockercfg-b4f6m\""
Jul 25 01:09:56.568539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.568499    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:09:56.615940 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.615908    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.616740 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.616437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.616740 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.616570    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnh8\" (UniqueName: \"kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.622186 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.619071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.698060 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.698038    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:56.717099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717067    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717114    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717162    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717203    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717240    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717269    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pj7q\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717306    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717344    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717383    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:09:56.717686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo\") pod \"8b689f77-e600-4bc4-974d-c5a8da9471bf\" (UID: \"8b689f77-e600-4bc4-974d-c5a8da9471bf\") "
Jul 25 01:09:56.717742 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.717798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnh8\" (UniqueName: \"kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.718895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717991    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.718895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.717987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:09:56.718895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.718375    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:09:56.718895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.718489    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:09:56.718895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.718683    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:09:56.719256 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.719239    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.720492 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.720466    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:09:56.720711 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.720619    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:09:56.720849 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.720818    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q" (OuterVolumeSpecName: "kube-api-access-8pj7q") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "kube-api-access-8pj7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:09:56.720965 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.720879    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b689f77-e600-4bc4-974d-c5a8da9471bf" (UID: "8b689f77-e600-4bc4-974d-c5a8da9471bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:09:56.725458 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.725436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnh8\" (UniqueName: \"kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8\") pod \"payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819298    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819321    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819331    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pj7q\" (UniqueName: \"kubernetes.io/projected/8b689f77-e600-4bc4-974d-c5a8da9471bf-kube-api-access-8pj7q\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819340    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819348    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819356    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819365 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819363    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b689f77-e600-4bc4-974d-c5a8da9471bf-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.819703 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.819372    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b689f77-e600-4bc4-974d-c5a8da9471bf-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:09:56.875271 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.875228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:56.875436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:56.875299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:57.040169 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:09:57.040133    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a37573_39da_4798_b973_8f765cff37db.slice/crio-ed4578cb29e33811fb455cb8dc2a73be328bd7e61d565f2595de5bdc2e76b7e1 WatchSource:0}: Error finding container ed4578cb29e33811fb455cb8dc2a73be328bd7e61d565f2595de5bdc2e76b7e1: Status 404 returned error can't find the container with id ed4578cb29e33811fb455cb8dc2a73be328bd7e61d565f2595de5bdc2e76b7e1
Jul 25 01:09:57.040368 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.040349    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:09:57.061231 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.061202    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:09:57.062061 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:09:57.062033    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53be8ca_f382_43f4_84f1_40b686a5ca9d.slice/crio-525e1e3aa7434c5c82ea6d15cf53d69c3e78295c64d209a1c107d5438e46577f WatchSource:0}: Error finding container 525e1e3aa7434c5c82ea6d15cf53d69c3e78295c64d209a1c107d5438e46577f: Status 404 returned error can't find the container with id 525e1e3aa7434c5c82ea6d15cf53d69c3e78295c64d209a1c107d5438e46577f
Jul 25 01:09:57.063574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.063554    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:09:57.493122 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.493088    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" event={"ID":"95a37573-39da-4798-b973-8f765cff37db","Type":"ContainerStarted","Data":"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"}
Jul 25 01:09:57.493295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.493149    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" event={"ID":"95a37573-39da-4798-b973-8f765cff37db","Type":"ContainerStarted","Data":"ed4578cb29e33811fb455cb8dc2a73be328bd7e61d565f2595de5bdc2e76b7e1"}
Jul 25 01:09:57.493295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.493181    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:09:57.494807 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.494781    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerID="7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12" exitCode=0
Jul 25 01:09:57.494942 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.494813    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerDied","Data":"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"}
Jul 25 01:09:57.494942 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.494846    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"
Jul 25 01:09:57.494942 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.494867    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7" event={"ID":"8b689f77-e600-4bc4-974d-c5a8da9471bf","Type":"ContainerDied","Data":"a51aaaab5fffa0fd7eb689b4419857e5b4468246a296dd8059159061d69b74e4"}
Jul 25 01:09:57.494942 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.494898    2574 scope.go:117] "RemoveContainer" containerID="7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"
Jul 25 01:09:57.496279 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.496255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" event={"ID":"d53be8ca-f382-43f4-84f1-40b686a5ca9d","Type":"ContainerStarted","Data":"525e1e3aa7434c5c82ea6d15cf53d69c3e78295c64d209a1c107d5438e46577f"}
Jul 25 01:09:57.505286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.505272    2574 scope.go:117] "RemoveContainer" containerID="a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"
Jul 25 01:09:57.509502 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.509456    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" podStartSLOduration=1.509439459 podStartE2EDuration="1.509439459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:09:57.506925339 +0000 UTC m=+2052.218343740" watchObservedRunningTime="2026-07-25 01:09:57.509439459 +0000 UTC m=+2052.220857858"
Jul 25 01:09:57.515262 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.515245    2574 scope.go:117] "RemoveContainer" containerID="7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"
Jul 25 01:09:57.515538 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:09:57.515501    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12\": container with ID starting with 7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12 not found: ID does not exist" containerID="7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"
Jul 25 01:09:57.515596 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.515550    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12"} err="failed to get container status \"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12\": rpc error: code = NotFound desc = could not find container \"7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12\": container with ID starting with 7857c01f145f8b1c8ed0330753fc452113296a3747e6854e64fb35e8971ddb12 not found: ID does not exist"
Jul 25 01:09:57.515596 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.515573    2574 scope.go:117] "RemoveContainer" containerID="a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"
Jul 25 01:09:57.515839 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:09:57.515820    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4\": container with ID starting with a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4 not found: ID does not exist" containerID="a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"
Jul 25 01:09:57.515887 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.515849    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4"} err="failed to get container status \"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4\": rpc error: code = NotFound desc = could not find container \"a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4\": container with ID starting with a424c4721e70bfd77f61d39502e84388733daf2a74be64d20ae5c1d362c4cff4 not found: ID does not exist"
Jul 25 01:09:57.520949 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.520930    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:57.523492 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.523473    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-82216ebf-openshift-default-54754c59d4-f2cl7"]
Jul 25 01:09:57.935411 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:57.935323    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" path="/var/lib/kubelet/pods/8b689f77-e600-4bc4-974d-c5a8da9471bf/volumes"
Jul 25 01:09:59.512312 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:59.512280    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" event={"ID":"d53be8ca-f382-43f4-84f1-40b686a5ca9d","Type":"ContainerStarted","Data":"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"}
Jul 25 01:09:59.512753 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:59.512411    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:09:59.528669 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:09:59.528627    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" podStartSLOduration=3.110830701 podStartE2EDuration="4.528613666s" totalImagesPullingTime="1.417782965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:09:59.527320926 +0000 UTC m=+2054.238739324" watchObservedRunningTime="2026-07-25 01:09:59.528613666 +0000 UTC m=+2054.240032064"
Jul 25 01:10:05.525565 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:05.525533    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:10:08.507421 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:08.507347    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:10:30.876075 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.876035    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:10:30.876950 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.876679    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" podUID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" containerName="maas-api" containerID="cri-o://a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab" gracePeriod=30
Jul 25 01:10:30.908267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.905958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:30.940856 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.940824    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:10:30.941107 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.941068    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" podUID="95a37573-39da-4798-b973-8f765cff37db" containerName="payload-processing" containerID="cri-o://310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2" gracePeriod=30
Jul 25 01:10:30.999016 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:30.998966    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:31.098120 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.098081    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:31.217181 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.217155    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:10:31.220819 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.220797    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:10:31.331442 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331410    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") pod \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") "
Jul 25 01:10:31.331637 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331464    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca\") pod \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") "
Jul 25 01:10:31.331637 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331489    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt2ph\" (UniqueName: \"kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph\") pod \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\" (UID: \"d53be8ca-f382-43f4-84f1-40b686a5ca9d\") "
Jul 25 01:10:31.331637 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331588    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume\") pod \"95a37573-39da-4798-b973-8f765cff37db\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") "
Jul 25 01:10:31.331799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbnh8\" (UniqueName: \"kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8\") pod \"95a37573-39da-4798-b973-8f765cff37db\" (UID: \"95a37573-39da-4798-b973-8f765cff37db\") "
Jul 25 01:10:31.331989 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.331924    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d53be8ca-f382-43f4-84f1-40b686a5ca9d" (UID: "d53be8ca-f382-43f4-84f1-40b686a5ca9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:10:31.332117 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.332014    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95a37573-39da-4798-b973-8f765cff37db" (UID: "95a37573-39da-4798-b973-8f765cff37db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:10:31.332117 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.332022    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d53be8ca-f382-43f4-84f1-40b686a5ca9d-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:10:31.334207 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.334172    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d53be8ca-f382-43f4-84f1-40b686a5ca9d" (UID: "d53be8ca-f382-43f4-84f1-40b686a5ca9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:10:31.334305 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.334206    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8" (OuterVolumeSpecName: "kube-api-access-zbnh8") pod "95a37573-39da-4798-b973-8f765cff37db" (UID: "95a37573-39da-4798-b973-8f765cff37db"). InnerVolumeSpecName "kube-api-access-zbnh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:10:31.334491 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.334472    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph" (OuterVolumeSpecName: "kube-api-access-lt2ph") pod "d53be8ca-f382-43f4-84f1-40b686a5ca9d" (UID: "d53be8ca-f382-43f4-84f1-40b686a5ca9d"). InnerVolumeSpecName "kube-api-access-lt2ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:10:31.433074 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.433007    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d53be8ca-f382-43f4-84f1-40b686a5ca9d-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:10:31.433074 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.433040    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt2ph\" (UniqueName: \"kubernetes.io/projected/d53be8ca-f382-43f4-84f1-40b686a5ca9d-kube-api-access-lt2ph\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:10:31.433074 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.433055    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95a37573-39da-4798-b973-8f765cff37db-plugins-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:10:31.433074 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.433069    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbnh8\" (UniqueName: \"kubernetes.io/projected/95a37573-39da-4798-b973-8f765cff37db-kube-api-access-zbnh8\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:10:31.711855 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.711819    2574 generic.go:356] "Generic (PLEG): container finished" podID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" containerID="a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab" exitCode=0
Jul 25 01:10:31.712017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.711860    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"
Jul 25 01:10:31.712017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.711903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" event={"ID":"d53be8ca-f382-43f4-84f1-40b686a5ca9d","Type":"ContainerDied","Data":"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"}
Jul 25 01:10:31.712017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.711952    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm" event={"ID":"d53be8ca-f382-43f4-84f1-40b686a5ca9d","Type":"ContainerDied","Data":"525e1e3aa7434c5c82ea6d15cf53d69c3e78295c64d209a1c107d5438e46577f"}
Jul 25 01:10:31.712017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.711975    2574 scope.go:117] "RemoveContainer" containerID="a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"
Jul 25 01:10:31.713674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.713648    2574 generic.go:356] "Generic (PLEG): container finished" podID="95a37573-39da-4798-b973-8f765cff37db" containerID="310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2" exitCode=0
Jul 25 01:10:31.713787 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.713700    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"
Jul 25 01:10:31.713787 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.713716    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" event={"ID":"95a37573-39da-4798-b973-8f765cff37db","Type":"ContainerDied","Data":"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"}
Jul 25 01:10:31.713787 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.713753    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th" event={"ID":"95a37573-39da-4798-b973-8f765cff37db","Type":"ContainerDied","Data":"ed4578cb29e33811fb455cb8dc2a73be328bd7e61d565f2595de5bdc2e76b7e1"}
Jul 25 01:10:31.724265 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.724247    2574 scope.go:117] "RemoveContainer" containerID="a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"
Jul 25 01:10:31.724498 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:10:31.724482    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab\": container with ID starting with a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab not found: ID does not exist" containerID="a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"
Jul 25 01:10:31.724578 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.724505    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab"} err="failed to get container status \"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab\": rpc error: code = NotFound desc = could not find container \"a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab\": container with ID starting with a5cc3400f6c4a2747f710aed5bc7f3e4ec48866cb681bf14b187ea613b4e6aab not found: ID does not exist"
Jul 25 01:10:31.724578 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.724539    2574 scope.go:117] "RemoveContainer" containerID="310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"
Jul 25 01:10:31.733043 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.733025    2574 scope.go:117] "RemoveContainer" containerID="310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"
Jul 25 01:10:31.733281 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:10:31.733260    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2\": container with ID starting with 310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2 not found: ID does not exist" containerID="310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"
Jul 25 01:10:31.733328 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.733290    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2"} err="failed to get container status \"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2\": rpc error: code = NotFound desc = could not find container \"310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2\": container with ID starting with 310ebc8c4a59bfbb915111166c1fe315f75f4d77e20b2d78db532f81c2e8d4f2 not found: ID does not exist"
Jul 25 01:10:31.746820 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.746800    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:10:31.751702 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.751677    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6fd8e2-5947d647f8-smpgm"]
Jul 25 01:10:31.765222 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.765201    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:10:31.771890 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.771871    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6fd8e2-5fdfc76cd5-h64th"]
Jul 25 01:10:31.933492 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.933452    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a37573-39da-4798-b973-8f765cff37db" path="/var/lib/kubelet/pods/95a37573-39da-4798-b973-8f765cff37db/volumes"
Jul 25 01:10:31.934033 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:31.934011    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" path="/var/lib/kubelet/pods/d53be8ca-f382-43f4-84f1-40b686a5ca9d/volumes"
Jul 25 01:10:35.527229 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:35.527189    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:35.687334 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:35.687303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:35.926181 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:35.926091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:36.155564 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.155499    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:36.449159 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449125    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:10:36.449812 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449795    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" containerName="maas-api"
Jul 25 01:10:36.449870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449816    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" containerName="maas-api"
Jul 25 01:10:36.449870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449838    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:36.449870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449847    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:36.449968 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449869    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a37573-39da-4798-b973-8f765cff37db" containerName="payload-processing"
Jul 25 01:10:36.449968 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449878    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a37573-39da-4798-b973-8f765cff37db" containerName="payload-processing"
Jul 25 01:10:36.450034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.449997    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d53be8ca-f382-43f4-84f1-40b686a5ca9d" containerName="maas-api"
Jul 25 01:10:36.450034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.450010    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:36.450034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.450019    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a37573-39da-4798-b973-8f765cff37db" containerName="payload-processing"
Jul 25 01:10:36.455246 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.455223    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.457637 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.457614    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0f061979-openshift-default-dockercfg-7swdp\""
Jul 25 01:10:36.463894 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.463872    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:10:36.478780 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.478712    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:36.579674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rk4l\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.579951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.580010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.580113 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.580075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.680902 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.680870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rk4l\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.680922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681214 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681214 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681216    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681569 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681589    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681695 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681555    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.681996 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.681919    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.682255 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.682232    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.683865 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.683834    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.684064 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.684047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.688823 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.688797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.688963 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.688943    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rk4l\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l\") pod \"e2e-ait-0f061979-openshift-default-86c7484984-5rqxd\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") " pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.770364 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.770341    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:36.782658 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.782612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:36.911831 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.911802    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:10:36.912637 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:10:36.912611    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8bbb17_9b7b_4c01_822e_f74dd6ed0a0d.slice/crio-112f3214724a7efc34a9806d12c1adb9d10dbd54e692ea6de79cbbefd3ea9db5 WatchSource:0}: Error finding container 112f3214724a7efc34a9806d12c1adb9d10dbd54e692ea6de79cbbefd3ea9db5: Status 404 returned error can't find the container with id 112f3214724a7efc34a9806d12c1adb9d10dbd54e692ea6de79cbbefd3ea9db5
Jul 25 01:10:36.914908 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.914871    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:36.915020 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.914954    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:36.915020 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:36.915001    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:37.756898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:37.756865    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerStarted","Data":"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"}
Jul 25 01:10:37.757264 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:37.756914    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerStarted","Data":"112f3214724a7efc34a9806d12c1adb9d10dbd54e692ea6de79cbbefd3ea9db5"}
Jul 25 01:10:37.770619 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:37.770599    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:37.775568 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:37.775546    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:37.779437 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:37.779398    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" podStartSLOduration=1.779384012 podStartE2EDuration="1.779384012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:10:37.776776986 +0000 UTC m=+2092.488195398" watchObservedRunningTime="2026-07-25 01:10:37.779384012 +0000 UTC m=+2092.490802411"
Jul 25 01:10:38.763257 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:38.763218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:38.764219 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:38.764200    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:39.108829 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.108751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:39.211693 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.211659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:39.289503 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.289473    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:10:39.289962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.289948    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:39.289962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.289963    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:39.290090 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.290059    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b689f77-e600-4bc4-974d-c5a8da9471bf" containerName="istio-proxy"
Jul 25 01:10:39.293441 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.293421    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.296953 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.296930    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0f061979-serving-cert\""
Jul 25 01:10:39.298026 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.298008    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9j2xk\""
Jul 25 01:10:39.305949 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.305931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.306035 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.305978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55d7z\" (UniqueName: \"kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.306083 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.306039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.318261 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.318236    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:10:39.358630 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.358599    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:39.407376 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.407308    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.407536 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.407432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.407536 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:10:39.407482    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-serving-cert: secret "maas-api-e2e-ait-0f061979-serving-cert" not found
Jul 25 01:10:39.407673 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.407577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55d7z\" (UniqueName: \"kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.407673 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:10:39.407594    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls podName:10274a70-677f-4fd7-a110-fa2e51e6a227 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:39.907570038 +0000 UTC m=+2094.618988435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls") pod "maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" (UID: "10274a70-677f-4fd7-a110-fa2e51e6a227") : secret "maas-api-e2e-ait-0f061979-serving-cert" not found
Jul 25 01:10:39.408141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.408124    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.417623 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.417599    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55d7z\" (UniqueName: \"kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.643399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.643363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:39.860350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.860319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:39.911387 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.911355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.913742 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.913717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") pod \"maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:39.961835 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:39.961803    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:40.207122 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:40.207085    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:40.362670 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:40.362630    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:10:40.363877 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:10:40.363847    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10274a70_677f_4fd7_a110_fa2e51e6a227.slice/crio-cf3e995c9963c5a295b073db279e71818152f4e4c0f15969e0236aa365232741 WatchSource:0}: Error finding container cf3e995c9963c5a295b073db279e71818152f4e4c0f15969e0236aa365232741: Status 404 returned error can't find the container with id cf3e995c9963c5a295b073db279e71818152f4e4c0f15969e0236aa365232741
Jul 25 01:10:40.777413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:40.777382    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" event={"ID":"10274a70-677f-4fd7-a110-fa2e51e6a227","Type":"ContainerStarted","Data":"cf3e995c9963c5a295b073db279e71818152f4e4c0f15969e0236aa365232741"}
Jul 25 01:10:41.784712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:41.784675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" event={"ID":"10274a70-677f-4fd7-a110-fa2e51e6a227","Type":"ContainerStarted","Data":"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"}
Jul 25 01:10:41.785090 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:41.784786    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:41.802911 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:41.802869    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" podStartSLOduration=2.391969022 podStartE2EDuration="2.802855739s" totalImagesPullingTime="410.886717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:10:41.801790933 +0000 UTC m=+2096.513209331" watchObservedRunningTime="2026-07-25 01:10:41.802855739 +0000 UTC m=+2096.514274137"
Jul 25 01:10:46.001542 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:46.001495    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:10:46.011057 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:46.011035    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:10:47.795729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:47.795698    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:10:49.505148 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:49.505115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:49.607012 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:49.606968    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:52.701137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.701099    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:52.760503 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.760466    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:10:52.862784 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.862748    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerID="8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4" exitCode=0
Jul 25 01:10:52.862898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.862817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerDied","Data":"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"}
Jul 25 01:10:52.863261 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.863245    2574 scope.go:117] "RemoveContainer" containerID="8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"
Jul 25 01:10:52.864337 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.864304    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:52.864420 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.864374    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:52.864420 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:52.864414    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:10:53.764063 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:53.764030    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:53.870728 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:53.870684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerStarted","Data":"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"}
Jul 25 01:10:54.770627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:54.770588    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:54.771908 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:54.771880    2574 patch_prober.go:28] interesting pod/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 25 01:10:54.772031 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:54.771926    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 25 01:10:55.771176 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:55.771139    2574 patch_prober.go:28] interesting pod/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 25 01:10:55.771620 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:55.771198    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 25 01:10:56.770907 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:56.770869    2574 patch_prober.go:28] interesting pod/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 25 01:10:56.771086 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:56.770956    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 25 01:10:57.775232 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:57.775199    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:57.775650 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:57.775463    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:10:57.776134 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:10:57.776114    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:11:15.379882 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.379842    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:11:15.380468 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.380131    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" podUID="10274a70-677f-4fd7-a110-fa2e51e6a227" containerName="maas-api" containerID="cri-o://7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d" gracePeriod=30
Jul 25 01:11:15.407483 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.407450    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:15.468428 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.468387    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:15.531188 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.531156    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:15.646009 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.645946    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:11:15.778045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.778011    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55d7z\" (UniqueName: \"kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z\") pod \"10274a70-677f-4fd7-a110-fa2e51e6a227\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") "
Jul 25 01:11:15.778045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.778049    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") pod \"10274a70-677f-4fd7-a110-fa2e51e6a227\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") "
Jul 25 01:11:15.778301 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.778082    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca\") pod \"10274a70-677f-4fd7-a110-fa2e51e6a227\" (UID: \"10274a70-677f-4fd7-a110-fa2e51e6a227\") "
Jul 25 01:11:15.778625 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.778597    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10274a70-677f-4fd7-a110-fa2e51e6a227" (UID: "10274a70-677f-4fd7-a110-fa2e51e6a227"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:11:15.780248 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.780223    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10274a70-677f-4fd7-a110-fa2e51e6a227" (UID: "10274a70-677f-4fd7-a110-fa2e51e6a227"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:11:15.780367 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.780348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z" (OuterVolumeSpecName: "kube-api-access-55d7z") pod "10274a70-677f-4fd7-a110-fa2e51e6a227" (UID: "10274a70-677f-4fd7-a110-fa2e51e6a227"). InnerVolumeSpecName "kube-api-access-55d7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:11:15.879221 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.879185    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55d7z\" (UniqueName: \"kubernetes.io/projected/10274a70-677f-4fd7-a110-fa2e51e6a227-kube-api-access-55d7z\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:15.879221 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.879216    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10274a70-677f-4fd7-a110-fa2e51e6a227-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:15.879221 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:15.879225    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10274a70-677f-4fd7-a110-fa2e51e6a227-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:16.003902 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.003870    2574 generic.go:356] "Generic (PLEG): container finished" podID="10274a70-677f-4fd7-a110-fa2e51e6a227" containerID="7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d" exitCode=0
Jul 25 01:11:16.004072 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.003918    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"
Jul 25 01:11:16.004072 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.003951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" event={"ID":"10274a70-677f-4fd7-a110-fa2e51e6a227","Type":"ContainerDied","Data":"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"}
Jul 25 01:11:16.004072 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.004006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf" event={"ID":"10274a70-677f-4fd7-a110-fa2e51e6a227","Type":"ContainerDied","Data":"cf3e995c9963c5a295b073db279e71818152f4e4c0f15969e0236aa365232741"}
Jul 25 01:11:16.004072 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.004029    2574 scope.go:117] "RemoveContainer" containerID="7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"
Jul 25 01:11:16.013362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.013344    2574 scope.go:117] "RemoveContainer" containerID="7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"
Jul 25 01:11:16.013662 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:16.013641    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d\": container with ID starting with 7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d not found: ID does not exist" containerID="7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"
Jul 25 01:11:16.013737 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.013669    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d"} err="failed to get container status \"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d\": rpc error: code = NotFound desc = could not find container \"7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d\": container with ID starting with 7c8441dcef7c1761cecb95834a7796603ae5f071485b5698526435bc54e31b8d not found: ID does not exist"
Jul 25 01:11:16.022976 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.022955    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:11:16.025778 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:16.025757    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0f061979-bc6c9dc98-wcncf"]
Jul 25 01:11:17.934959 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:17.934922    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10274a70-677f-4fd7-a110-fa2e51e6a227" path="/var/lib/kubelet/pods/10274a70-677f-4fd7-a110-fa2e51e6a227/volumes"
Jul 25 01:11:20.067809 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:20.067772    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:21.282848 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:21.282814    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:11:21.283338 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:21.283023    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy" containerID="cri-o://e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54" gracePeriod=30
Jul 25 01:11:21.298838 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:21.298811    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:21.548886 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:21.548785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:21.780803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:21.780771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:22.083016 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:22.082965    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:22.560541 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:22.560489    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:24.734412 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.734354    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:24.819769 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.819733    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:11:24.820254 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.820235    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10274a70-677f-4fd7-a110-fa2e51e6a227" containerName="maas-api"
Jul 25 01:11:24.820254 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.820251    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="10274a70-677f-4fd7-a110-fa2e51e6a227" containerName="maas-api"
Jul 25 01:11:24.820446 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.820344    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="10274a70-677f-4fd7-a110-fa2e51e6a227" containerName="maas-api"
Jul 25 01:11:24.821366 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.821340    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:24.823780 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.823762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.826727 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.826706    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9j2xk\""
Jul 25 01:11:24.826843 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.826785    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a271f0bb-serving-cert\""
Jul 25 01:11:24.838768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.838744    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:11:24.863003 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.862970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.863181 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.863038    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92677\" (UniqueName: \"kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.863181 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.863118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.900873 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.900839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:24.964727 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.964683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.964956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.964748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92677\" (UniqueName: \"kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.964956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.964778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.964956 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:24.964801    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-serving-cert: secret "maas-api-e2e-derive-a271f0bb-serving-cert" not found
Jul 25 01:11:24.964956 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:24.964878    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls podName:c8de5548-e1ad-400f-b328-3ab22d1667d9 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:25.464856922 +0000 UTC m=+2140.176275322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls") pod "maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" (UID: "c8de5548-e1ad-400f-b328-3ab22d1667d9") : secret "maas-api-e2e-derive-a271f0bb-serving-cert" not found
Jul 25 01:11:24.965467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.965448    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:24.973656 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:24.973628    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92677\" (UniqueName: \"kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:25.468725 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.468690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:25.468902 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:25.468845    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-serving-cert: secret "maas-api-e2e-derive-a271f0bb-serving-cert" not found
Jul 25 01:11:25.468956 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:25.468926    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls podName:c8de5548-e1ad-400f-b328-3ab22d1667d9 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:26.468906036 +0000 UTC m=+2141.180324427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls") pod "maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" (UID: "c8de5548-e1ad-400f-b328-3ab22d1667d9") : secret "maas-api-e2e-derive-a271f0bb-serving-cert" not found
Jul 25 01:11:25.515940 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.515900    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:25.615687 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.615653    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:25.630188 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.630157    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:11:25.634118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.634092    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.636369 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.636343    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a271f0bb\""
Jul 25 01:11:25.644727 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.644704    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:11:25.669712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.669684    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:11:25.671144 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.671118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhp2k\" (UniqueName: \"kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.671284 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.671266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.673478 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.673463    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.676154 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.676134    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a271f0bb-dockercfg-b2nwf\""
Jul 25 01:11:25.685961 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.685931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:11:25.701263 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.701227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:11:25.772058 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.771982    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6nw8\" (UniqueName: \"kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.772058 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.772036    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.772459 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.772100    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.772459 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.772161    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhp2k\" (UniqueName: \"kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.772781 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.772764    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.780024 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.780001    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhp2k\" (UniqueName: \"kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k\") pod \"payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.873398 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.873355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6nw8\" (UniqueName: \"kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.873624 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.873451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.874170 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.874148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.886062 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.886034    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6nw8\" (UniqueName: \"kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8\") pod \"payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") " pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:25.947966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.947933    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:25.985456 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:25.985414    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:26.319331 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.319306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:11:26.344768 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:11:26.344722    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ebd9ef_8cf8_4194_bcd3_9e074b422420.slice/crio-e99af05f4a6eb018c71f3f27c6950fad3563f2287e0c3b2abf07105ecadeec3b WatchSource:0}: Error finding container e99af05f4a6eb018c71f3f27c6950fad3563f2287e0c3b2abf07105ecadeec3b: Status 404 returned error can't find the container with id e99af05f4a6eb018c71f3f27c6950fad3563f2287e0c3b2abf07105ecadeec3b
Jul 25 01:11:26.356101 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.354640    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:11:26.480076 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.480040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:26.482633 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.482575    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:26.533559 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.533490    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:11:26.586605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586436    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.586605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.586605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586554    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.586605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586579    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.586605 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586604    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.587008 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rk4l\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.587008 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586753    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:11:26.587008 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.586929    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.587180 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.587078    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:11:26.587334 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.587310    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:11:26.587837 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.587811    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:11:26.589477 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.589425    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:11:26.589826 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.589803    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l" (OuterVolumeSpecName: "kube-api-access-4rk4l") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "kube-api-access-4rk4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:11:26.637450 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.637402    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687315    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687366    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687595    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo\") pod \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\" (UID: \"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d\") "
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687948    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687968    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687983    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.687997    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rk4l\" (UniqueName: \"kubernetes.io/projected/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-kube-api-access-4rk4l\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.688010    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.688283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.688010    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:11:26.690825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.690772    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:11:26.690825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.690814    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" (UID: "7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:11:26.789349 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.789316    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.789349 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.789347    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.789774 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.789359    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:11:26.800220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:26.800190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:11:26.801805 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:11:26.801776    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8de5548_e1ad_400f_b328_3ab22d1667d9.slice/crio-46fbd677988376083e652aa08b254d9e32b78d03a4c81b53843d3c7abdd92baa WatchSource:0}: Error finding container 46fbd677988376083e652aa08b254d9e32b78d03a4c81b53843d3c7abdd92baa: Status 404 returned error can't find the container with id 46fbd677988376083e652aa08b254d9e32b78d03a4c81b53843d3c7abdd92baa
Jul 25 01:11:27.080479 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.080445    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerID="e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54" exitCode=0
Jul 25 01:11:27.080689 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.080494    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"
Jul 25 01:11:27.080689 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.080555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerDied","Data":"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"}
Jul 25 01:11:27.080689 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.080622    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd" event={"ID":"7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d","Type":"ContainerDied","Data":"112f3214724a7efc34a9806d12c1adb9d10dbd54e692ea6de79cbbefd3ea9db5"}
Jul 25 01:11:27.080689 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.080652    2574 scope.go:117] "RemoveContainer" containerID="e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"
Jul 25 01:11:27.082128 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.081951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" event={"ID":"c8de5548-e1ad-400f-b328-3ab22d1667d9","Type":"ContainerStarted","Data":"46fbd677988376083e652aa08b254d9e32b78d03a4c81b53843d3c7abdd92baa"}
Jul 25 01:11:27.083802 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.083775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" event={"ID":"7a64e815-8138-4dff-830a-2024d47740de","Type":"ContainerStarted","Data":"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"}
Jul 25 01:11:27.083922 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.083822    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" event={"ID":"7a64e815-8138-4dff-830a-2024d47740de","Type":"ContainerStarted","Data":"8d96e9818e3ecb0a42f0b24e99c925d66f6e7753421e06c811e2e6448830161a"}
Jul 25 01:11:27.083922 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.083885    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:27.085704 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.085683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" event={"ID":"85ebd9ef-8cf8-4194-bcd3-9e074b422420","Type":"ContainerStarted","Data":"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"}
Jul 25 01:11:27.085809 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.085719    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" event={"ID":"85ebd9ef-8cf8-4194-bcd3-9e074b422420","Type":"ContainerStarted","Data":"e99af05f4a6eb018c71f3f27c6950fad3563f2287e0c3b2abf07105ecadeec3b"}
Jul 25 01:11:27.085809 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.085744    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:11:27.092273 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.092260    2574 scope.go:117] "RemoveContainer" containerID="8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"
Jul 25 01:11:27.101185 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.101133    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" podStartSLOduration=2.101117329 podStartE2EDuration="2.101117329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:27.099292049 +0000 UTC m=+2141.810710451" watchObservedRunningTime="2026-07-25 01:11:27.101117329 +0000 UTC m=+2141.812535729"
Jul 25 01:11:27.103382 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.103364    2574 scope.go:117] "RemoveContainer" containerID="e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"
Jul 25 01:11:27.103640 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:27.103623    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54\": container with ID starting with e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54 not found: ID does not exist" containerID="e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"
Jul 25 01:11:27.103695 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.103648    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54"} err="failed to get container status \"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54\": rpc error: code = NotFound desc = could not find container \"e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54\": container with ID starting with e3077569aee5587532336c40dbfeab4371a200b20d9f48d382c00b26fd009e54 not found: ID does not exist"
Jul 25 01:11:27.103695 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.103663    2574 scope.go:117] "RemoveContainer" containerID="8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"
Jul 25 01:11:27.103943 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:11:27.103925    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4\": container with ID starting with 8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4 not found: ID does not exist" containerID="8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"
Jul 25 01:11:27.103982 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.103948    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4"} err="failed to get container status \"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4\": rpc error: code = NotFound desc = could not find container \"8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4\": container with ID starting with 8800ff5917f9cfe4017baf45c6a981fa6c16c3a613ce33e4afe5ab154a98abd4 not found: ID does not exist"
Jul 25 01:11:27.112948 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.112926    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:11:27.116851 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.116833    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0f061979-openshift-default-86c7484984-5rqxd"]
Jul 25 01:11:27.127922 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.127883    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" podStartSLOduration=2.127869374 podStartE2EDuration="2.127869374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:27.126733085 +0000 UTC m=+2141.838151495" watchObservedRunningTime="2026-07-25 01:11:27.127869374 +0000 UTC m=+2141.839287778"
Jul 25 01:11:27.934426 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:27.934391    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" path="/var/lib/kubelet/pods/7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d/volumes"
Jul 25 01:11:28.093801 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:28.093758    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" event={"ID":"c8de5548-e1ad-400f-b328-3ab22d1667d9","Type":"ContainerStarted","Data":"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"}
Jul 25 01:11:28.094061 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:28.094038    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:28.112167 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:28.112116    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" podStartSLOduration=3.58427052 podStartE2EDuration="4.112101486s" totalImagesPullingTime="527.830966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:28.108661521 +0000 UTC m=+2142.820079920" watchObservedRunningTime="2026-07-25 01:11:28.112101486 +0000 UTC m=+2142.823519886"
Jul 25 01:11:34.106147 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:34.106119    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:11:38.095751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:38.095709    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:11:38.096177 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:11:38.096114    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:12:00.212962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.212925    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:12:00.213622 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.213186    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" podUID="c8de5548-e1ad-400f-b328-3ab22d1667d9" containerName="maas-api" containerID="cri-o://776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3" gracePeriod=30
Jul 25 01:12:00.245396 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.245364    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:00.276944 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.276914    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:12:00.277146 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.277122    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" podUID="7a64e815-8138-4dff-830a-2024d47740de" containerName="payload-processing" containerID="cri-o://256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3" gracePeriod=30
Jul 25 01:12:00.284770 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.284740    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:12:00.285023 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.284997    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" podUID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" containerName="payload-pre-processing" containerID="cri-o://9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a" gracePeriod=30
Jul 25 01:12:00.318557 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.318357    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:00.383567 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.383530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:00.569771 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.569741    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:12:00.584988 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.584964    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:12:00.589706 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.589684    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:12:00.728204 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728164    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume\") pod \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728226    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca\") pod \"c8de5548-e1ad-400f-b328-3ab22d1667d9\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728251    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhp2k\" (UniqueName: \"kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k\") pod \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\" (UID: \"85ebd9ef-8cf8-4194-bcd3-9e074b422420\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728283    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") pod \"c8de5548-e1ad-400f-b328-3ab22d1667d9\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728307    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92677\" (UniqueName: \"kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677\") pod \"c8de5548-e1ad-400f-b328-3ab22d1667d9\" (UID: \"c8de5548-e1ad-400f-b328-3ab22d1667d9\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728352    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6nw8\" (UniqueName: \"kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8\") pod \"7a64e815-8138-4dff-830a-2024d47740de\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") "
Jul 25 01:12:00.728404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume\") pod \"7a64e815-8138-4dff-830a-2024d47740de\" (UID: \"7a64e815-8138-4dff-830a-2024d47740de\") "
Jul 25 01:12:00.728751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8de5548-e1ad-400f-b328-3ab22d1667d9" (UID: "c8de5548-e1ad-400f-b328-3ab22d1667d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:12:00.728751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728633    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85ebd9ef-8cf8-4194-bcd3-9e074b422420" (UID: "85ebd9ef-8cf8-4194-bcd3-9e074b422420"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:12:00.728751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728736    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85ebd9ef-8cf8-4194-bcd3-9e074b422420-plugins-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.728880 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728761    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8de5548-e1ad-400f-b328-3ab22d1667d9-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.728989 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.728964    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a64e815-8138-4dff-830a-2024d47740de" (UID: "7a64e815-8138-4dff-830a-2024d47740de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:12:00.730593 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.730548    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8de5548-e1ad-400f-b328-3ab22d1667d9" (UID: "c8de5548-e1ad-400f-b328-3ab22d1667d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:12:00.730699 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.730599    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k" (OuterVolumeSpecName: "kube-api-access-zhp2k") pod "85ebd9ef-8cf8-4194-bcd3-9e074b422420" (UID: "85ebd9ef-8cf8-4194-bcd3-9e074b422420"). InnerVolumeSpecName "kube-api-access-zhp2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:00.730855 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.730834    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677" (OuterVolumeSpecName: "kube-api-access-92677") pod "c8de5548-e1ad-400f-b328-3ab22d1667d9" (UID: "c8de5548-e1ad-400f-b328-3ab22d1667d9"). InnerVolumeSpecName "kube-api-access-92677". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:00.731014 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.730994    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8" (OuterVolumeSpecName: "kube-api-access-t6nw8") pod "7a64e815-8138-4dff-830a-2024d47740de" (UID: "7a64e815-8138-4dff-830a-2024d47740de"). InnerVolumeSpecName "kube-api-access-t6nw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:00.829881 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.829795    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6nw8\" (UniqueName: \"kubernetes.io/projected/7a64e815-8138-4dff-830a-2024d47740de-kube-api-access-t6nw8\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.829881 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.829832    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a64e815-8138-4dff-830a-2024d47740de-plugins-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.829881 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.829843    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhp2k\" (UniqueName: \"kubernetes.io/projected/85ebd9ef-8cf8-4194-bcd3-9e074b422420-kube-api-access-zhp2k\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.829881 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.829853    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8de5548-e1ad-400f-b328-3ab22d1667d9-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:00.829881 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:00.829861    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92677\" (UniqueName: \"kubernetes.io/projected/c8de5548-e1ad-400f-b328-3ab22d1667d9-kube-api-access-92677\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:12:01.305504 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.305471    2574 generic.go:356] "Generic (PLEG): container finished" podID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" containerID="9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a" exitCode=0
Jul 25 01:12:01.305966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.305526    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"
Jul 25 01:12:01.305966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.305550    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" event={"ID":"85ebd9ef-8cf8-4194-bcd3-9e074b422420","Type":"ContainerDied","Data":"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"}
Jul 25 01:12:01.305966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.305606    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz" event={"ID":"85ebd9ef-8cf8-4194-bcd3-9e074b422420","Type":"ContainerDied","Data":"e99af05f4a6eb018c71f3f27c6950fad3563f2287e0c3b2abf07105ecadeec3b"}
Jul 25 01:12:01.305966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.305628    2574 scope.go:117] "RemoveContainer" containerID="9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"
Jul 25 01:12:01.307278 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.307242    2574 generic.go:356] "Generic (PLEG): container finished" podID="c8de5548-e1ad-400f-b328-3ab22d1667d9" containerID="776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3" exitCode=0
Jul 25 01:12:01.307462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.307291    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"
Jul 25 01:12:01.307462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.307326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" event={"ID":"c8de5548-e1ad-400f-b328-3ab22d1667d9","Type":"ContainerDied","Data":"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"}
Jul 25 01:12:01.307462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.307380    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54" event={"ID":"c8de5548-e1ad-400f-b328-3ab22d1667d9","Type":"ContainerDied","Data":"46fbd677988376083e652aa08b254d9e32b78d03a4c81b53843d3c7abdd92baa"}
Jul 25 01:12:01.308814 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.308788    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a64e815-8138-4dff-830a-2024d47740de" containerID="256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3" exitCode=0
Jul 25 01:12:01.308971 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.308840    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"
Jul 25 01:12:01.308971 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.308875    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" event={"ID":"7a64e815-8138-4dff-830a-2024d47740de","Type":"ContainerDied","Data":"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"}
Jul 25 01:12:01.308971 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.308918    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2" event={"ID":"7a64e815-8138-4dff-830a-2024d47740de","Type":"ContainerDied","Data":"8d96e9818e3ecb0a42f0b24e99c925d66f6e7753421e06c811e2e6448830161a"}
Jul 25 01:12:01.317641 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.317624    2574 scope.go:117] "RemoveContainer" containerID="9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"
Jul 25 01:12:01.317946 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:12:01.317921    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a\": container with ID starting with 9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a not found: ID does not exist" containerID="9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"
Jul 25 01:12:01.318018 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.317958    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a"} err="failed to get container status \"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a\": rpc error: code = NotFound desc = could not find container \"9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a\": container with ID starting with 9547e3234f4e142093762768ffee564c9c138b6d8071bf1bf2d47098cba5a10a not found: ID does not exist"
Jul 25 01:12:01.318018 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.317983    2574 scope.go:117] "RemoveContainer" containerID="776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"
Jul 25 01:12:01.326918 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.326901    2574 scope.go:117] "RemoveContainer" containerID="776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"
Jul 25 01:12:01.327158 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:12:01.327139    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3\": container with ID starting with 776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3 not found: ID does not exist" containerID="776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"
Jul 25 01:12:01.327226 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.327162    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3"} err="failed to get container status \"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3\": rpc error: code = NotFound desc = could not find container \"776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3\": container with ID starting with 776d283667e6419c83c086f79721717ba8ded26b5c7ef50730ca9147f6c327a3 not found: ID does not exist"
Jul 25 01:12:01.327226 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.327179    2574 scope.go:117] "RemoveContainer" containerID="256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"
Jul 25 01:12:01.333291 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.333271    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:12:01.337340 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.337321    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a271f0bb-75db688c76-dv6sz"]
Jul 25 01:12:01.337952 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.337933    2574 scope.go:117] "RemoveContainer" containerID="256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"
Jul 25 01:12:01.338224 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:12:01.338204    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3\": container with ID starting with 256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3 not found: ID does not exist" containerID="256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"
Jul 25 01:12:01.338308 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.338230    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3"} err="failed to get container status \"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3\": rpc error: code = NotFound desc = could not find container \"256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3\": container with ID starting with 256d4915a0a4a585a87694b72ab846c31ebc7a570aac27d0ae3575230b8028a3 not found: ID does not exist"
Jul 25 01:12:01.347791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.347756    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:12:01.352592 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.352571    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a271f0bb-6cfcd465cb-dgz54"]
Jul 25 01:12:01.361619 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.361601    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:12:01.365242 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.365219    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a271f0bb-7bc7bfff86-rssq2"]
Jul 25 01:12:01.934350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.934316    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a64e815-8138-4dff-830a-2024d47740de" path="/var/lib/kubelet/pods/7a64e815-8138-4dff-830a-2024d47740de/volumes"
Jul 25 01:12:01.934728 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.934713    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" path="/var/lib/kubelet/pods/85ebd9ef-8cf8-4194-bcd3-9e074b422420/volumes"
Jul 25 01:12:01.935066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:01.935054    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8de5548-e1ad-400f-b328-3ab22d1667d9" path="/var/lib/kubelet/pods/c8de5548-e1ad-400f-b328-3ab22d1667d9/volumes"
Jul 25 01:12:04.723933 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:04.723897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:04.881781 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:04.881752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:05.146027 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:05.145907    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:05.650307 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:05.650256    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:05.746642 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:05.746607    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:05.846399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:05.846356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:07.385912 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:07.385863    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:12.978200 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:12.978154    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:13.045186 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:13.045135    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:13.123089 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:13.123056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:13.218845 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:13.218807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:13.320141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:13.320046    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:13.713098 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:13.713066    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:32.641335 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:32.641296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:39.709927 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:39.709892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:46.055352 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:46.055313    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:46.156120 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:46.156088    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:46.253819 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:46.253780    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:46.362457 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:12:46.362379    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:02.255990 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:02.255961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:07.957826 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:07.957785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:08.058909 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:08.058864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:08.158604 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:08.158532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:18.754865 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:18.754834    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:19.560720 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:19.560686    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:25.848783 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:25.848745    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:25.969593 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:25.969552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:26.075377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:26.075346    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:26.279690 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:26.279651    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:27.021417 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:27.021381    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:27.115665 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:27.115603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:27.217451 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:27.217416    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:28.834918 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:28.834873    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:34.895811 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:34.895774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:34.995966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:34.995879    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:35.585711 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:35.585675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:41.364902 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.364867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:41.875828 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.875800    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:13:41.876286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876272    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876289    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876301    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8de5548-e1ad-400f-b328-3ab22d1667d9" containerName="maas-api"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876308    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8de5548-e1ad-400f-b328-3ab22d1667d9" containerName="maas-api"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876325    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" containerName="payload-pre-processing"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876331    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" containerName="payload-pre-processing"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876339    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a64e815-8138-4dff-830a-2024d47740de" containerName="payload-processing"
Jul 25 01:13:41.876351 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876344    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a64e815-8138-4dff-830a-2024d47740de" containerName="payload-processing"
Jul 25 01:13:41.876589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876424    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:41.876589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876431    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a64e815-8138-4dff-830a-2024d47740de" containerName="payload-processing"
Jul 25 01:13:41.876589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876439    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:41.876589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876445    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ebd9ef-8cf8-4194-bcd3-9e074b422420" containerName="payload-pre-processing"
Jul 25 01:13:41.876589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.876452    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8de5548-e1ad-400f-b328-3ab22d1667d9" containerName="maas-api"
Jul 25 01:13:41.879682 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.879664    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.882118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.882082    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-818dfc55-openshift-default-dockercfg-2r6hf\""
Jul 25 01:13:41.899296 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.899272    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:13:41.978669 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.978863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.978863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-477x8\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.978863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.978863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978832    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.979093 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.979093 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.978977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.979093 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.979050    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:41.979093 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:41.979080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.079903 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.079867    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.079930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-477x8\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.079968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.079992    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080081 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080315    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080581 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080556    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080670 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080633    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.080943 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.080925    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.082524 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.082487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.082651 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.082576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.087438 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.087419    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.087655 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.087638    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-477x8\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8\") pod \"e2e-mt-818dfc55-openshift-default-5899f65d68-9472d\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") " pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.191645 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.191555    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:42.350150 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.350122    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:13:42.351357 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:13:42.351333    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20bf3cce_47bc_4074_b499_47132b6e03ba.slice/crio-a5902323e020f423f456d93e4a228653e1b4067ff37ff9c56a094061945a677e WatchSource:0}: Error finding container a5902323e020f423f456d93e4a228653e1b4067ff37ff9c56a094061945a677e: Status 404 returned error can't find the container with id a5902323e020f423f456d93e4a228653e1b4067ff37ff9c56a094061945a677e
Jul 25 01:13:42.353550 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.353494    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:42.353623 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.353578    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:42.353623 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.353606    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:42.797871 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.797837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:42.958047 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.958004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerStarted","Data":"7cc0c61ca8dd120976b7b8a4e3a65481b0f80cc25eb5b78ae4603111d4d823f2"}
Jul 25 01:13:42.958250 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.958064    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerStarted","Data":"a5902323e020f423f456d93e4a228653e1b4067ff37ff9c56a094061945a677e"}
Jul 25 01:13:42.980683 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:42.980624    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podStartSLOduration=1.980607451 podStartE2EDuration="1.980607451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:42.977373443 +0000 UTC m=+2277.688791843" watchObservedRunningTime="2026-07-25 01:13:42.980607451 +0000 UTC m=+2277.692025850"
Jul 25 01:13:43.192308 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:43.192216    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:43.197292 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:43.197263    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:43.964335 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:43.964305    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:43.965364 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:43.965345    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:44.916328 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:44.916290    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.004688 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.004638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.075164 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.075127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.222883 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.222853    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.715116 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.715085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.824579 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.824537    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:45.835631 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.835602    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:13:45.836127 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.836107    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:45.836127 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.836127    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8bbb17-9b7b-4c01-822e-f74dd6ed0a0d" containerName="istio-proxy"
Jul 25 01:13:45.839706 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.839688    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:45.842174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.842152    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-818dfc55-dockercfg-7cpsk\""
Jul 25 01:13:45.842278 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.842171    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-818dfc55\""
Jul 25 01:13:45.866221 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.866182    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:13:45.916720 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.916683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:45.916884 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:45.916728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9qbh\" (UniqueName: \"kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.018460 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.018349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.018460 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.018417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9qbh\" (UniqueName: \"kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.019141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.019117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.026798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.026771    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9qbh\" (UniqueName: \"kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh\") pod \"payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.152048 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.152015    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:46.320782 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.320739    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:13:46.322346 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:13:46.322317    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3cdeb3_96ee_49f0_9fd5_19a35f9b9927.slice/crio-b8f44171e87b3a1e29edac77ee06dc51bff423f057a1d8992172b13362987d8c WatchSource:0}: Error finding container b8f44171e87b3a1e29edac77ee06dc51bff423f057a1d8992172b13362987d8c: Status 404 returned error can't find the container with id b8f44171e87b3a1e29edac77ee06dc51bff423f057a1d8992172b13362987d8c
Jul 25 01:13:46.988635 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.988598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" event={"ID":"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927","Type":"ContainerStarted","Data":"09fdb4a0e170534aa1134cfe8c7600ca1269605d949fa22fbcc8495e5197fbb9"}
Jul 25 01:13:46.988834 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.988658    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" event={"ID":"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927","Type":"ContainerStarted","Data":"b8f44171e87b3a1e29edac77ee06dc51bff423f057a1d8992172b13362987d8c"}
Jul 25 01:13:46.988999 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:46.988963    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:47.008243 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:47.008194    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" podStartSLOduration=2.008180472 podStartE2EDuration="2.008180472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:47.005944708 +0000 UTC m=+2281.717363110" watchObservedRunningTime="2026-07-25 01:13:47.008180472 +0000 UTC m=+2281.719598870"
Jul 25 01:13:49.427403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:49.427369    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:50.102497 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:50.102458    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:50.212455 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:50.212431    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:50.570597 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:50.570561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:56.198298 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:56.198258    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:56.299146 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:56.299113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:57.997195 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:57.997112    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:13:58.057624 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.057584    2574 generic.go:356] "Generic (PLEG): container finished" podID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerID="7cc0c61ca8dd120976b7b8a4e3a65481b0f80cc25eb5b78ae4603111d4d823f2" exitCode=0
Jul 25 01:13:58.057810 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.057671    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerDied","Data":"7cc0c61ca8dd120976b7b8a4e3a65481b0f80cc25eb5b78ae4603111d4d823f2"}
Jul 25 01:13:58.058071 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.058057    2574 scope.go:117] "RemoveContainer" containerID="7cc0c61ca8dd120976b7b8a4e3a65481b0f80cc25eb5b78ae4603111d4d823f2"
Jul 25 01:13:58.059067 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.059041    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:58.059125 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.059106    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:58.059161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.059138    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:13:58.964408 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:58.964380    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:59.065390 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:59.065347    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerStarted","Data":"95d7c35f2dc8f8f228693d9ccf594afd35d1a8ee8ec27e581dfee6527f57189d"}
Jul 25 01:13:59.192327 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:59.192299    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:13:59.193111 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:59.193086    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Jul 25 01:13:59.193224 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:13:59.193139    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Jul 25 01:14:00.192488 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:00.192457    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Jul 25 01:14:00.192885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:00.192538    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Jul 25 01:14:01.192053 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:01.192019    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Jul 25 01:14:01.192230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:01.192079    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Jul 25 01:14:02.192296 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:02.192261    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Jul 25 01:14:02.192716 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:02.192321    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Jul 25 01:14:03.192331 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:03.192293    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Jul 25 01:14:03.192844 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:03.192355    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Jul 25 01:14:04.196408 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:04.196378    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:14:04.196883 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:04.196861    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:14:04.197500 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:04.197482    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:14:06.957420 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:06.957380    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:06.982662 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:06.982632    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:14:06.982879 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:06.982850    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" podUID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" containerName="payload-pre-processing" containerID="cri-o://09fdb4a0e170534aa1134cfe8c7600ca1269605d949fa22fbcc8495e5197fbb9" gracePeriod=30
Jul 25 01:14:07.057310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.057271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:07.115799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.115761    2574 generic.go:356] "Generic (PLEG): container finished" podID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" containerID="09fdb4a0e170534aa1134cfe8c7600ca1269605d949fa22fbcc8495e5197fbb9" exitCode=0
Jul 25 01:14:07.115945 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.115831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" event={"ID":"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927","Type":"ContainerDied","Data":"09fdb4a0e170534aa1134cfe8c7600ca1269605d949fa22fbcc8495e5197fbb9"}
Jul 25 01:14:07.258805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.258773    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:14:07.319141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.319116    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume\") pod \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") "
Jul 25 01:14:07.319326 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.319195    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9qbh\" (UniqueName: \"kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh\") pod \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\" (UID: \"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927\") "
Jul 25 01:14:07.319580 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.319538    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" (UID: "7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.321201 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.321181    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh" (OuterVolumeSpecName: "kube-api-access-z9qbh") pod "7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" (UID: "7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927"). InnerVolumeSpecName "kube-api-access-z9qbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:07.419841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.419810    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9qbh\" (UniqueName: \"kubernetes.io/projected/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-kube-api-access-z9qbh\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.419841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:07.419838    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927-plugins-config-volume\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:08.122719 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:08.122684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2" event={"ID":"7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927","Type":"ContainerDied","Data":"b8f44171e87b3a1e29edac77ee06dc51bff423f057a1d8992172b13362987d8c"}
Jul 25 01:14:08.123149 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:08.122725    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"
Jul 25 01:14:08.123149 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:08.122753    2574 scope.go:117] "RemoveContainer" containerID="09fdb4a0e170534aa1134cfe8c7600ca1269605d949fa22fbcc8495e5197fbb9"
Jul 25 01:14:08.143107 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:08.143060    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:14:08.146359 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:08.146335    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-818dfc55-579b6487f6-wxxd2"]
Jul 25 01:14:09.934505 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:09.934465    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" path="/var/lib/kubelet/pods/7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927/volumes"
Jul 25 01:14:11.643278 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:11.643248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:12.042672 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:12.042639    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:17.755690 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:17.755657    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:17.857803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:17.857771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:18.015142 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.015059    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:14:18.015310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.015283    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" containerID="cri-o://95d7c35f2dc8f8f228693d9ccf594afd35d1a8ee8ec27e581dfee6527f57189d" gracePeriod=30
Jul 25 01:14:18.038608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.038571    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:18.277495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.277407    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:18.451161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.451112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:18.699968 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.699866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:18.813296 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:18.813269    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:19.199028 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:19.198991    2574 patch_prober.go:28] interesting pod/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 25 01:14:19.199213 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:19.199054    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 01:14:23.221743 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.221710    2574 generic.go:356] "Generic (PLEG): container finished" podID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerID="95d7c35f2dc8f8f228693d9ccf594afd35d1a8ee8ec27e581dfee6527f57189d" exitCode=0
Jul 25 01:14:23.222136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.221782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerDied","Data":"95d7c35f2dc8f8f228693d9ccf594afd35d1a8ee8ec27e581dfee6527f57189d"}
Jul 25 01:14:23.222136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.221847    2574 scope.go:117] "RemoveContainer" containerID="7cc0c61ca8dd120976b7b8a4e3a65481b0f80cc25eb5b78ae4603111d4d823f2"
Jul 25 01:14:23.281194 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.281135    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:14:23.372982 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.372954    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373120 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373007    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373120 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373036    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373120 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373071    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373151    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373176    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373265    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-477x8\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8\") pod \"20bf3cce-47bc-4074-b499-47132b6e03ba\" (UID: \"20bf3cce-47bc-4074-b499-47132b6e03ba\") "
Jul 25 01:14:23.373597 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373330    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:23.373597 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373470    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:23.373597 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373486    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:23.373762 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373636    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:23.373762 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373737    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20bf3cce-47bc-4074-b499-47132b6e03ba-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.373891 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373768    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.373891 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373783    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.373891 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.373798    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.374380 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.374352    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:23.375478 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.375454    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:23.375610 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.375524    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8" (OuterVolumeSpecName: "kube-api-access-477x8") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "kube-api-access-477x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:23.375686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.375667    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:14:23.375740 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.375688    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20bf3cce-47bc-4074-b499-47132b6e03ba" (UID: "20bf3cce-47bc-4074-b499-47132b6e03ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:23.474432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.474396    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.474432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.474423    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.474639 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.474437    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-477x8\" (UniqueName: \"kubernetes.io/projected/20bf3cce-47bc-4074-b499-47132b6e03ba-kube-api-access-477x8\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.474639 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.474452    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:23.474639 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:23.474464    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20bf3cce-47bc-4074-b499-47132b6e03ba-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:14:24.228825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.228795    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"
Jul 25 01:14:24.229211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.228836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d" event={"ID":"20bf3cce-47bc-4074-b499-47132b6e03ba","Type":"ContainerDied","Data":"a5902323e020f423f456d93e4a228653e1b4067ff37ff9c56a094061945a677e"}
Jul 25 01:14:24.229211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.228896    2574 scope.go:117] "RemoveContainer" containerID="95d7c35f2dc8f8f228693d9ccf594afd35d1a8ee8ec27e581dfee6527f57189d"
Jul 25 01:14:24.250119 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.250093    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:14:24.254654 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.254634    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-818dfc55-openshift-default-5899f65d68-9472d"]
Jul 25 01:14:24.676597 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.676460    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:24.798347 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.798316    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:24.900771 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:24.900736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:25.088263 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:25.088227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:25.934344 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:25.934307    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" path="/var/lib/kubelet/pods/20bf3cce-47bc-4074-b499-47132b6e03ba/volumes"
Jul 25 01:14:30.696824 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:30.696787    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:30.753824 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:30.753795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:37.538849 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.538816    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:37.860336 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.860259    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:14:37.860986 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.860966    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:14:37.860986 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.860986    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:14:37.861108 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861007    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:14:37.861108 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861016    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:14:37.861108 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861028    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" containerName="payload-pre-processing"
Jul 25 01:14:37.861108 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861037    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" containerName="payload-pre-processing"
Jul 25 01:14:37.861247 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861161    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3cdeb3-96ee-49f0-9fd5-19a35f9b9927" containerName="payload-pre-processing"
Jul 25 01:14:37.861247 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.861175    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:14:37.865944 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.865924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:37.868370 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.868346    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-98d19c4d-openshift-default-dockercfg-zxc65\""
Jul 25 01:14:37.877789 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.875776    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:14:37.886957 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:37.886826    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:38.017818 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.017784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.017818 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.017823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.017892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.017956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.017976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.018001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vslsz\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018276 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.018083    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018276 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.018135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.018276 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.018171    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119234 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vslsz\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119234 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119192    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119234 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119487    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119859    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119965 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.119965 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.119904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.120245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.120226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.121699 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.121676    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.122107 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.122090    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.126835 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.126812    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.127162 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.127141    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vslsz\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz\") pod \"e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") " pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.188652 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.188588    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:38.290969 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.290892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:38.340883 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.340853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:14:38.342365 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:14:38.342339    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod242e7649_9d46_482f_8196_2160e9c0cc54.slice/crio-7eaf39ca3a2ce5c6ccd0bf9f45b5c072c5f325ada1c8af8fef4119afe86ec2aa WatchSource:0}: Error finding container 7eaf39ca3a2ce5c6ccd0bf9f45b5c072c5f325ada1c8af8fef4119afe86ec2aa: Status 404 returned error can't find the container with id 7eaf39ca3a2ce5c6ccd0bf9f45b5c072c5f325ada1c8af8fef4119afe86ec2aa
Jul 25 01:14:38.344646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.344613    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:38.344737 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.344679    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:38.344737 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:38.344724    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:39.322543 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:39.322487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerStarted","Data":"071ffe127e86686a571d710fe6d3b4454308844b1bc178cc0f38b61cc928896e"}
Jul 25 01:14:39.322938 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:39.322566    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerStarted","Data":"7eaf39ca3a2ce5c6ccd0bf9f45b5c072c5f325ada1c8af8fef4119afe86ec2aa"}
Jul 25 01:14:39.344783 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:39.344733    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podStartSLOduration=2.344717307 podStartE2EDuration="2.344717307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:14:39.343299163 +0000 UTC m=+2334.054717575" watchObservedRunningTime="2026-07-25 01:14:39.344717307 +0000 UTC m=+2334.056135705"
Jul 25 01:14:40.188775 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:40.188741    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:40.193601 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:40.193577    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:40.328548 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:40.328498    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:40.329344 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:40.329327    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:41.224105 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:41.224074    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:41.327692 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:41.327655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:41.451315 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:41.451285    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:41.779707 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:41.779670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:41.911808 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:41.911774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:42.023052 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:42.023013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:45.750777 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:45.750746    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:56.076037 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.074848    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": read tcp 10.132.0.2:39150->10.132.0.82:15021: read: connection reset by peer" start-of-body=
Jul 25 01:14:56.076037 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.074922    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": read tcp 10.132.0.2:39150->10.132.0.82:15021: read: connection reset by peer"
Jul 25 01:14:56.430676 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.430572    2574 generic.go:356] "Generic (PLEG): container finished" podID="242e7649-9d46-482f-8196-2160e9c0cc54" containerID="071ffe127e86686a571d710fe6d3b4454308844b1bc178cc0f38b61cc928896e" exitCode=0
Jul 25 01:14:56.430676 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.430641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerDied","Data":"071ffe127e86686a571d710fe6d3b4454308844b1bc178cc0f38b61cc928896e"}
Jul 25 01:14:56.431078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.431062    2574 scope.go:117] "RemoveContainer" containerID="071ffe127e86686a571d710fe6d3b4454308844b1bc178cc0f38b61cc928896e"
Jul 25 01:14:56.432203 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.432172    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:56.432300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.432242    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:56.432300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:56.432273    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:14:57.443318 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:57.443270    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerStarted","Data":"09c8d281685f785dfeb9a55a5da88d3e9aa5d1eb77e3f07efa78f7e391fd23c0"}
Jul 25 01:14:58.189289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:58.189254    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:14:58.189462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:58.189432    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 25 01:14:58.189532 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:58.189487    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 25 01:14:59.189427 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:59.189389    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 25 01:14:59.189852 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:14:59.189446    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 25 01:15:00.130673 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.130638    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"]
Jul 25 01:15:00.131184 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.131170    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20bf3cce-47bc-4074-b499-47132b6e03ba" containerName="istio-proxy"
Jul 25 01:15:00.135397 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.135363    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"
Jul 25 01:15:00.139214 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.139189    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:15:00.141989 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.141961    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"]
Jul 25 01:15:00.172527 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.169551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"
Jul 25 01:15:00.188978 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.188940    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 25 01:15:00.189099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.189007    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 25 01:15:00.359952 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.359638    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"]
Jul 25 01:15:00.362758 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:15:00.362718    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415c9bbc_b711_4ec3_a883_7e303bc49803.slice/crio-95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d WatchSource:0}: Error finding container 95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d: Status 404 returned error can't find the container with id 95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d
Jul 25 01:15:00.468816 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:00.468782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq" event={"ID":"415c9bbc-b711-4ec3-a883-7e303bc49803","Type":"ContainerStarted","Data":"95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d"}
Jul 25 01:15:01.189275 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:01.189239    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 25 01:15:01.189433 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:01.189291    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 25 01:15:01.476466 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:01.476432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq" event={"ID":"415c9bbc-b711-4ec3-a883-7e303bc49803","Type":"ContainerStarted","Data":"671e99a55383a95a57d7a40eebfbb791d5fb854afc4f26c2adaf37b068b32e88"}
Jul 25 01:15:01.493415 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:01.493369    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq" podStartSLOduration=1.493356777 podStartE2EDuration="1.493356777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:15:01.491014714 +0000 UTC m=+2356.202433114" watchObservedRunningTime="2026-07-25 01:15:01.493356777 +0000 UTC m=+2356.204775175"
Jul 25 01:15:02.193220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:02.193186    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:15:02.193455 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:02.193437    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:15:02.194246 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:02.194227    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:15:02.485029 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:02.484992    2574 generic.go:356] "Generic (PLEG): container finished" podID="415c9bbc-b711-4ec3-a883-7e303bc49803" containerID="671e99a55383a95a57d7a40eebfbb791d5fb854afc4f26c2adaf37b068b32e88" exitCode=0
Jul 25 01:15:02.485480 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:02.485056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq" event={"ID":"415c9bbc-b711-4ec3-a883-7e303bc49803","Type":"ContainerDied","Data":"671e99a55383a95a57d7a40eebfbb791d5fb854afc4f26c2adaf37b068b32e88"}
Jul 25 01:15:03.637201 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:03.637177    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"
Jul 25 01:15:04.505191 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:04.505163    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq"
Jul 25 01:15:04.505361 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:04.505184    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749035-vf6dq" event={"ID":"415c9bbc-b711-4ec3-a883-7e303bc49803","Type":"ContainerDied","Data":"95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d"}
Jul 25 01:15:04.505361 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:04.505256    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95019a72919c1b7ec89e752b0b87d4f7af54976094af76347ca82292b3033b0d"
Jul 25 01:15:07.923996 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:07.923947    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:41.597805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:41.597766    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:41.648432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:41.648398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:46.048187 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:46.048158    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:15:46.060949 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:46.060923    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:15:51.209121 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:51.209089    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:51.278491 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:51.278453    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:51.334735 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:51.334703    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:55.782973 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:55.782942    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:57.166136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:57.166105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:57.264382 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:57.264349    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:59.172519 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:15:59.172476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:04.248076 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:04.248038    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:40.293803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:40.293770    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:41.273895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:41.273862    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:46.781526 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:46.781485    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:46.839461 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:46.839427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:47.045078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.044984    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:16:47.045652 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.045617    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" containerID="cri-o://09c8d281685f785dfeb9a55a5da88d3e9aa5d1eb77e3f07efa78f7e391fd23c0" gracePeriod=30
Jul 25 01:16:47.060886 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.060855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:47.196086 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.196050    2574 patch_prober.go:28] interesting pod/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 25 01:16:47.196240 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.196111    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 01:16:47.296933 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:47.296843    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:48.404718 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:48.404676    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:48.497406 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:48.497358    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:48.654177 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:48.654140    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:49.347628 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:49.347586    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:49.446145 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:49.446113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:49.549070 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:49.549032    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:50.997784 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:50.997732    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:52.179424 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.179393    2574 generic.go:356] "Generic (PLEG): container finished" podID="242e7649-9d46-482f-8196-2160e9c0cc54" containerID="09c8d281685f785dfeb9a55a5da88d3e9aa5d1eb77e3f07efa78f7e391fd23c0" exitCode=0
Jul 25 01:16:52.179819 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.179473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerDied","Data":"09c8d281685f785dfeb9a55a5da88d3e9aa5d1eb77e3f07efa78f7e391fd23c0"}
Jul 25 01:16:52.179819 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.179566    2574 scope.go:117] "RemoveContainer" containerID="071ffe127e86686a571d710fe6d3b4454308844b1bc178cc0f38b61cc928896e"
Jul 25 01:16:52.323185 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.323104    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:16:52.362069 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362029    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362100    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362159    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362188    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362216    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362286 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362277    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362598 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362327    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:16:52.362598 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362352    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:16:52.362598 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362338    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vslsz\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362598 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362401    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362598 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362479    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert\") pod \"242e7649-9d46-482f-8196-2160e9c0cc54\" (UID: \"242e7649-9d46-482f-8196-2160e9c0cc54\") "
Jul 25 01:16:52.362932 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362897    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.363010 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.362928    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.363220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.363189    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:16:52.363317 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.363215    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:16:52.364034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.363995    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data" (OuterVolumeSpecName: "istio-data") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:16:52.365267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.365240    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz" (OuterVolumeSpecName: "kube-api-access-vslsz") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "kube-api-access-vslsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:16:52.365439 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.365410    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token" (OuterVolumeSpecName: "istio-token") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:16:52.365594 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.365498    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:16:52.365773 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.365751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "242e7649-9d46-482f-8196-2160e9c0cc54" (UID: "242e7649-9d46-482f-8196-2160e9c0cc54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:16:52.463913 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463874    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.463913 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463904    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.463913 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463915    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.464206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463932    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242e7649-9d46-482f-8196-2160e9c0cc54-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.464206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463945    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vslsz\" (UniqueName: \"kubernetes.io/projected/242e7649-9d46-482f-8196-2160e9c0cc54-kube-api-access-vslsz\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.464206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463959    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242e7649-9d46-482f-8196-2160e9c0cc54-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:52.464206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:52.463972    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242e7649-9d46-482f-8196-2160e9c0cc54-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:16:53.187401 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.187369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm" event={"ID":"242e7649-9d46-482f-8196-2160e9c0cc54","Type":"ContainerDied","Data":"7eaf39ca3a2ce5c6ccd0bf9f45b5c072c5f325ada1c8af8fef4119afe86ec2aa"}
Jul 25 01:16:53.187885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.187429    2574 scope.go:117] "RemoveContainer" containerID="09c8d281685f785dfeb9a55a5da88d3e9aa5d1eb77e3f07efa78f7e391fd23c0"
Jul 25 01:16:53.187885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.187382    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"
Jul 25 01:16:53.213353 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.213317    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:16:53.217175 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.217148    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-98d19c4d-openshift-default-788b6d8756-b6bxm"]
Jul 25 01:16:53.933462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:53.933417    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" path="/var/lib/kubelet/pods/242e7649-9d46-482f-8196-2160e9c0cc54/volumes"
Jul 25 01:16:56.739032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:56.738997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:56.839358 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:56.839326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:57.446757 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:16:57.446720    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:00.772565 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:00.772527    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:03.175313 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:03.175279    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:03.281627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:03.281594    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:03.376460 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:03.376425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:13.900177 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:13.900146    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:14.285144 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:14.285111    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:35.311599 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:35.311559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:41.952075 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:41.951971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:42.066913 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.066878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:42.327640 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.327603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:42.625968 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.625883    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:17:42.626581 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626565    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:17:42.626642 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626584    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:17:42.626642 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626601    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="415c9bbc-b711-4ec3-a883-7e303bc49803" containerName="cleanup"
Jul 25 01:17:42.626642 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626610    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="415c9bbc-b711-4ec3-a883-7e303bc49803" containerName="cleanup"
Jul 25 01:17:42.626769 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626743    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:17:42.626769 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626760    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:17:42.626841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.626774    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="415c9bbc-b711-4ec3-a883-7e303bc49803" containerName="cleanup"
Jul 25 01:17:42.630403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.630386    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.633489 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.633466    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d08be1-openshift-default-dockercfg-9fsh9\""
Jul 25 01:17:42.642897 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.642869    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:17:42.651559 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.651530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:42.778030 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778030 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778258 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778258 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778071    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2qcm\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778258 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778197    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778289    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.778471 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.778399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.879825 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2qcm\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.880230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.880230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.879958    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.880230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.880148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.880230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.880225    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.880414 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.880392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.882026 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.882004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.882235 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.882218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.887551 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.887502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.887785 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.887765    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2qcm\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm\") pod \"e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") " pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:42.945330 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:42.945294    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:43.108298 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.107711    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:17:43.112319 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.112287    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:17:43.112387 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.112349    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:17:43.112387 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.112379    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:17:43.168273 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.168240    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:43.500753 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.500719    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerStarted","Data":"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"}
Jul 25 01:17:43.500926 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.500770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerStarted","Data":"1e1e5f6be6cbfae8259ed4d99e6c2b38687b31c8fc332c5cefa23708a5c73d29"}
Jul 25 01:17:43.523633 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.523586    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podStartSLOduration=1.5235713789999998 podStartE2EDuration="1.523571379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:43.522012108 +0000 UTC m=+2518.233430506" watchObservedRunningTime="2026-07-25 01:17:43.523571379 +0000 UTC m=+2518.234989777"
Jul 25 01:17:43.946372 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:43.946291    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:44.951240 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:44.951207    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:45.501353 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.501320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:45.523215 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.523182    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:45.524381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.524357    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:17:45.591753 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.591716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:45.666301 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.666265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:45.920909 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:45.920828    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:46.257436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:46.257406    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:46.354911 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:46.354874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:50.036084 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:50.036050    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:50.380916 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:50.380820    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:50.709665 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:50.709630    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:50.962368 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:50.962281    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.085899 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.085857    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.181161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.181085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.251404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.251368    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.557439 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.557345    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.795183 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.795149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:54.898316 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:54.898222    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:58.621663 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:17:58.621622    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:00.850782 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:00.849083    2574 patch_prober.go:28] interesting pod/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": read tcp 10.132.0.2:35422->10.132.0.84:15021: read: connection reset by peer" start-of-body=
Jul 25 01:18:00.850782 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:00.849177    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": read tcp 10.132.0.2:35422->10.132.0.84:15021: read: connection reset by peer"
Jul 25 01:18:01.629041 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.629004    2574 generic.go:356] "Generic (PLEG): container finished" podID="d9801872-8965-4eea-9658-9626c9e4d878" containerID="4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803" exitCode=0
Jul 25 01:18:01.629220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.629077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerDied","Data":"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"}
Jul 25 01:18:01.629501 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.629486    2574 scope.go:117] "RemoveContainer" containerID="4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"
Jul 25 01:18:01.630568 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.630537    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:18:01.630622 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.630610    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:18:01.630658 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:01.630644    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:18:02.637781 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:02.637749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerStarted","Data":"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"}
Jul 25 01:18:02.946551 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:02.946429    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:18:02.946743 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:02.946581    2574 patch_prober.go:28] interesting pod/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Jul 25 01:18:02.946743 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:02.946643    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Jul 25 01:18:03.946472 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:03.946426    2574 patch_prober.go:28] interesting pod/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Jul 25 01:18:03.947034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:03.946545    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Jul 25 01:18:04.946353 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:04.946314    2574 patch_prober.go:28] interesting pod/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Jul 25 01:18:04.946680 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:04.946640    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Jul 25 01:18:05.946769 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:05.946732    2574 patch_prober.go:28] interesting pod/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Jul 25 01:18:05.947332 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:05.947298    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Jul 25 01:18:06.950543 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:06.950487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:18:06.951031 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:06.950996    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:18:06.951706 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:06.951683    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:18:13.822154 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:13.822118    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:19.600556 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:19.600522    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:19.702099 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:19.702063    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:37.414651 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.414619    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:18:37.415114 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.415100    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:18:37.415160 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.415116    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="242e7649-9d46-482f-8196-2160e9c0cc54" containerName="istio-proxy"
Jul 25 01:18:37.417925 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.417910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.422802 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.422780    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9deb1f\"/\"kube-root-ca.crt\""
Jul 25 01:18:37.422802 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.422796    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9deb1f\"/\"test-model-9deb1f-kserve-self-signed-certs\""
Jul 25 01:18:37.422995 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.422809    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9deb1f\"/\"openshift-service-ca.crt\""
Jul 25 01:18:37.423882 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.423862    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9deb1f\"/\"default-dockercfg-nc8tj\""
Jul 25 01:18:37.435396 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.435373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:18:37.471153 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.471122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:37.510534 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.510686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.510686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.510686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.510820 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.510820 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.510807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhgzn\" (UniqueName: \"kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.611949 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.611911    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612138 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.611964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgzn\" (UniqueName: \"kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612138 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612138 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612138 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612138 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612130    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612421 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612485 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612444    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.612485 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.612464    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.614357 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.614335    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.614541 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.614523    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.620958 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.620939    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhgzn\" (UniqueName: \"kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn\") pod \"test-model-9deb1f-kserve-7b84f58fc9-dfxhc\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") " pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.727236 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.727202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:37.892372 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:37.892346    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:18:37.894272 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:18:37.894238    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59bc8abf_6062_485e_a5c1_4c24bc45617b.slice/crio-1b1a7925f72fc52b231aa00abd34cc4d29fd0b44bf97bf60a495aa93e4f77bb9 WatchSource:0}: Error finding container 1b1a7925f72fc52b231aa00abd34cc4d29fd0b44bf97bf60a495aa93e4f77bb9: Status 404 returned error can't find the container with id 1b1a7925f72fc52b231aa00abd34cc4d29fd0b44bf97bf60a495aa93e4f77bb9
Jul 25 01:18:38.897954 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:38.897917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" event={"ID":"59bc8abf-6062-485e-a5c1-4c24bc45617b","Type":"ContainerStarted","Data":"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"}
Jul 25 01:18:38.898367 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:38.897963    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" event={"ID":"59bc8abf-6062-485e-a5c1-4c24bc45617b","Type":"ContainerStarted","Data":"1b1a7925f72fc52b231aa00abd34cc4d29fd0b44bf97bf60a495aa93e4f77bb9"}
Jul 25 01:18:38.922030 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:38.921972    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" podStartSLOduration=1.9219557900000002 podStartE2EDuration="1.92195579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:38.920355487 +0000 UTC m=+2573.631773886" watchObservedRunningTime="2026-07-25 01:18:38.92195579 +0000 UTC m=+2573.633374189"
Jul 25 01:18:47.727789 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:47.727752    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:47.728381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:47.727808    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:47.743638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:47.743606    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:18:47.964793 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:18:47.964757    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:19:18.517627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:18.517542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:19.409538 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.409489    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:19:19.482471 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.482437    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:19.567318 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.567279    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"]
Jul 25 01:19:19.570687 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.570665    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:19.573306 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.573110    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:19:19.576413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.576388    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"]
Jul 25 01:19:19.617843 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.617815    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx\" (UID: \"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:19.718738 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.718713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx\" (UID: \"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:19.719402 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.719383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx\" (UID: \"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:19.885157 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:19.885130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:20.023863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:20.023841    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"]
Jul 25 01:19:20.026686 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:19:20.026656    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fb783d_8da7_4b93_9e00_4929b4cd2b9c.slice/crio-5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276 WatchSource:0}: Error finding container 5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276: Status 404 returned error can't find the container with id 5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276
Jul 25 01:19:20.168798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:20.168692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx" event={"ID":"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c","Type":"ContainerStarted","Data":"5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276"}
Jul 25 01:19:21.177582 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:21.177475    2574 generic.go:356] "Generic (PLEG): container finished" podID="c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" containerID="640b34f15ba0974768d72550281cb5318c6b69ca474c20da8c50c5efef647c18" exitCode=0
Jul 25 01:19:21.177582 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:21.177559    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx" event={"ID":"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c","Type":"ContainerDied","Data":"640b34f15ba0974768d72550281cb5318c6b69ca474c20da8c50c5efef647c18"}
Jul 25 01:19:22.325747 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:22.325723    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:22.447413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:22.447328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca\") pod \"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c\" (UID: \"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c\") "
Jul 25 01:19:22.447736 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:22.447713    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" (UID: "c3fb783d-8da7-4b93-9e00-4929b4cd2b9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:22.548285 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:22.548259    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c-maas-api-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:23.191423 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:23.191392    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"
Jul 25 01:19:23.191423 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:23.191417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx" event={"ID":"c3fb783d-8da7-4b93-9e00-4929b4cd2b9c","Type":"ContainerDied","Data":"5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276"}
Jul 25 01:19:23.191654 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:23.191465    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fc9748282456214f19831fecc430fed43a0684c27790c0ecf4a2e1f805e7276"
Jul 25 01:19:30.125232 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:30.125197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:30.224466 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:30.224432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:30.325629 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:30.325595    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:31.167619 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:31.167585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:31.224763 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:31.224730    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:34.428873 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.428819    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" podUID="59bc8abf-6062-485e-a5c1-4c24bc45617b" containerName="main" containerID="cri-o://cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054" gracePeriod=45
Jul 25 01:19:34.634780 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.634750    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:34.657724 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.657698    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"]
Jul 25 01:19:34.661036 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.661012    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d08be1-9llvx"]
Jul 25 01:19:34.691999 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.691951    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:19:34.763996 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.763970    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764119 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764018    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764119 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764101    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764232 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764150    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764232 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764184    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764232 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764215    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhgzn\" (UniqueName: \"kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn\") pod \"59bc8abf-6062-485e-a5c1-4c24bc45617b\" (UID: \"59bc8abf-6062-485e-a5c1-4c24bc45617b\") "
Jul 25 01:19:34.764380 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764299    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home" (OuterVolumeSpecName: "home") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:34.764380 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764362    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache" (OuterVolumeSpecName: "model-cache") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:34.764485 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764400    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:34.764688 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764659    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-home\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.764765 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764686    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-model-cache\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.764765 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.764700    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-tmp-dir\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.766266 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.766234    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:19:34.766360 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.766336    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn" (OuterVolumeSpecName: "kube-api-access-hhgzn") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "kube-api-access-hhgzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:34.766360 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.766337    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm" (OuterVolumeSpecName: "dshm") pod "59bc8abf-6062-485e-a5c1-4c24bc45617b" (UID: "59bc8abf-6062-485e-a5c1-4c24bc45617b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:34.865555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.865532    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhgzn\" (UniqueName: \"kubernetes.io/projected/59bc8abf-6062-485e-a5c1-4c24bc45617b-kube-api-access-hhgzn\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.865555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.865555    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59bc8abf-6062-485e-a5c1-4c24bc45617b-tls-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:34.865691 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:34.865564    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59bc8abf-6062-485e-a5c1-4c24bc45617b-dshm\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:35.000153 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.000130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:35.100117 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.100091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:35.273073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.273005    2574 generic.go:356] "Generic (PLEG): container finished" podID="59bc8abf-6062-485e-a5c1-4c24bc45617b" containerID="cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054" exitCode=0
Jul 25 01:19:35.273073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.273049    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"
Jul 25 01:19:35.273223 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.273092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" event={"ID":"59bc8abf-6062-485e-a5c1-4c24bc45617b","Type":"ContainerDied","Data":"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"}
Jul 25 01:19:35.273223 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.273135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc" event={"ID":"59bc8abf-6062-485e-a5c1-4c24bc45617b","Type":"ContainerDied","Data":"1b1a7925f72fc52b231aa00abd34cc4d29fd0b44bf97bf60a495aa93e4f77bb9"}
Jul 25 01:19:35.273223 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.273158    2574 scope.go:117] "RemoveContainer" containerID="cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"
Jul 25 01:19:35.283346 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.283328    2574 scope.go:117] "RemoveContainer" containerID="cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"
Jul 25 01:19:35.283608 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:19:35.283587    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054\": container with ID starting with cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054 not found: ID does not exist" containerID="cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"
Jul 25 01:19:35.283679 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.283615    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054"} err="failed to get container status \"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054\": rpc error: code = NotFound desc = could not find container \"cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054\": container with ID starting with cc8745b0dee45f748258b5467d60aeca369cac452d8cb8314687162d638c8054 not found: ID does not exist"
Jul 25 01:19:35.294756 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.294731    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:19:35.300361 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.300342    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9deb1f/test-model-9deb1f-kserve-7b84f58fc9-dfxhc"]
Jul 25 01:19:35.935339 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.935300    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59bc8abf-6062-485e-a5c1-4c24bc45617b" path="/var/lib/kubelet/pods/59bc8abf-6062-485e-a5c1-4c24bc45617b/volumes"
Jul 25 01:19:35.935880 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:35.935856    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" path="/var/lib/kubelet/pods/c3fb783d-8da7-4b93-9e00-4929b4cd2b9c/volumes"
Jul 25 01:19:40.943169 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:40.943130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:41.043539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:41.043495    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:41.232376 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:41.232341    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:19:41.232622 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:41.232598    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy" containerID="cri-o://2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117" gracePeriod=30
Jul 25 01:19:41.251757 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:41.251731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:41.457668 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:41.457634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:46.492518 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.492486    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:19:46.573324 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573290    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573342    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573369    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573391    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573418    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573434    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573550    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573586    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573625    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2qcm\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm\") pod \"d9801872-8965-4eea-9658-9626c9e4d878\" (UID: \"d9801872-8965-4eea-9658-9626c9e4d878\") "
Jul 25 01:19:46.573803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573758    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:46.574017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573836    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:46.574017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573927    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:46.574017 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.573965    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:46.574137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.574094    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.574137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.574114    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9801872-8965-4eea-9658-9626c9e4d878-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.574137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.574109    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:46.574137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.574127    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.574284 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.574141    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.575721 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.575699    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:19:46.575986 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.575968    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm" (OuterVolumeSpecName: "kube-api-access-s2qcm") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "kube-api-access-s2qcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:46.576126 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.576110    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:19:46.576189 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.576139    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9801872-8965-4eea-9658-9626c9e4d878" (UID: "d9801872-8965-4eea-9658-9626c9e4d878"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:46.674721 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.674690    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2qcm\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-kube-api-access-s2qcm\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.674721 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.674720    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.674925 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.674734    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9801872-8965-4eea-9658-9626c9e4d878-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.674925 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.674748    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9801872-8965-4eea-9658-9626c9e4d878-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:46.674925 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:46.674759    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9801872-8965-4eea-9658-9626c9e4d878-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:19:47.348535 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.348484    2574 generic.go:356] "Generic (PLEG): container finished" podID="d9801872-8965-4eea-9658-9626c9e4d878" containerID="2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117" exitCode=0
Jul 25 01:19:47.348726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.348545    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"
Jul 25 01:19:47.348726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.348566    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerDied","Data":"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"}
Jul 25 01:19:47.348726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.348623    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z" event={"ID":"d9801872-8965-4eea-9658-9626c9e4d878","Type":"ContainerDied","Data":"1e1e5f6be6cbfae8259ed4d99e6c2b38687b31c8fc332c5cefa23708a5c73d29"}
Jul 25 01:19:47.348726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.348646    2574 scope.go:117] "RemoveContainer" containerID="2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"
Jul 25 01:19:47.359241 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.359219    2574 scope.go:117] "RemoveContainer" containerID="4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"
Jul 25 01:19:47.367640 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.367624    2574 scope.go:117] "RemoveContainer" containerID="2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"
Jul 25 01:19:47.367888 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:19:47.367871    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117\": container with ID starting with 2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117 not found: ID does not exist" containerID="2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"
Jul 25 01:19:47.367972 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.367898    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117"} err="failed to get container status \"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117\": rpc error: code = NotFound desc = could not find container \"2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117\": container with ID starting with 2495964c15b7be9b4d29cc450164259c6492d5af41f6f59eee9f1182f6f13117 not found: ID does not exist"
Jul 25 01:19:47.367972 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.367922    2574 scope.go:117] "RemoveContainer" containerID="4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"
Jul 25 01:19:47.368351 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:19:47.368326    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803\": container with ID starting with 4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803 not found: ID does not exist" containerID="4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"
Jul 25 01:19:47.368466 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.368352    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803"} err="failed to get container status \"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803\": rpc error: code = NotFound desc = could not find container \"4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803\": container with ID starting with 4fd690991265ff4d26175f4c2b1640eadfb96afe0c0000ac826c768b6de48803 not found: ID does not exist"
Jul 25 01:19:47.372674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.372651    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:19:47.375381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.375359    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d08be1-openshift-default-7db7c5c8b-jbz6z"]
Jul 25 01:19:47.933715 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:47.933673    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9801872-8965-4eea-9658-9626c9e4d878" path="/var/lib/kubelet/pods/d9801872-8965-4eea-9658-9626c9e4d878/volumes"
Jul 25 01:19:52.038964 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:52.038931    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:52.146664 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:52.146627    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:52.244700 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:52.244668    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:56.662335 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:56.662301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:57.083303 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:19:57.083271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:02.719561 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:02.719500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:02.819832 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:02.819803    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:03.038160 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.038080    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:03.260089 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.260055    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:03.624277 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.624238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:03.922799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.922708    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:03.923239 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923223    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59bc8abf-6062-485e-a5c1-4c24bc45617b" containerName="main"
Jul 25 01:20:03.923239 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923238    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="59bc8abf-6062-485e-a5c1-4c24bc45617b" containerName="main"
Jul 25 01:20:03.923362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923277    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:03.923362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923283    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:03.923362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923296    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" containerName="revoke-keys"
Jul 25 01:20:03.923362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923302    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" containerName="revoke-keys"
Jul 25 01:20:03.923608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923370    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:03.923608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923378    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="59bc8abf-6062-485e-a5c1-4c24bc45617b" containerName="main"
Jul 25 01:20:03.923608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923385    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:03.923608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.923391    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3fb783d-8da7-4b93-9e00-4929b4cd2b9c" containerName="revoke-keys"
Jul 25 01:20:03.926617 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.926593    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:03.929310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.929281    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c59e95-openshift-default-dockercfg-nkx9z\""
Jul 25 01:20:03.941994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.941754    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:03.942228 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:03.941973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:04.026940 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.026902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027112 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.026967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027112 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.026997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027112 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46ff\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027268 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027268 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027374 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027374 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.027472 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.027389    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.128838 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46ff\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.128984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129019 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129416 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129416 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129323    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129660 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129630    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129746 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129686    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129807 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129764    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.129807 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.129785    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.131127 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.131107    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.131570 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.131544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.137078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.137050    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.137343 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.137322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46ff\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff\") pod \"e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") " pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.144903 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.144879    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:04.242292 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.242255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:04.390427 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.390397    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:04.391271 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:20:04.391245    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eff4ea7_7ad7_4acb_b6e8_61012ff98677.slice/crio-3cf29fd750e1006f1db1451d9753a7f2cb17bbb676a4d921b2b12b304a109a70 WatchSource:0}: Error finding container 3cf29fd750e1006f1db1451d9753a7f2cb17bbb676a4d921b2b12b304a109a70: Status 404 returned error can't find the container with id 3cf29fd750e1006f1db1451d9753a7f2cb17bbb676a4d921b2b12b304a109a70
Jul 25 01:20:04.393379 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.393341    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:04.393473 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.393410    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:04.393473 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.393442    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:04.458250 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.458222    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerStarted","Data":"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"}
Jul 25 01:20:04.458381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.458290    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerStarted","Data":"3cf29fd750e1006f1db1451d9753a7f2cb17bbb676a4d921b2b12b304a109a70"}
Jul 25 01:20:04.477894 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:04.477842    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podStartSLOduration=1.477824295 podStartE2EDuration="1.477824295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:04.476867139 +0000 UTC m=+2659.188285539" watchObservedRunningTime="2026-07-25 01:20:04.477824295 +0000 UTC m=+2659.189242695"
Jul 25 01:20:05.242527 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:05.242475    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:05.247804 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:05.247778    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:05.464561 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:05.464533    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:05.465549 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:05.465504    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:06.431697 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.431662    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:06.502747 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.502709    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:06.503190 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.503177    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:06.503245 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.503191    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9801872-8965-4eea-9658-9626c9e4d878" containerName="istio-proxy"
Jul 25 01:20:06.506601 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.506568    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.510272 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.510245    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c59e95-serving-cert\""
Jul 25 01:20:06.510411 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.510338    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9j2xk\""
Jul 25 01:20:06.522715 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.522687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:06.537947 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.537909    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:06.553054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.553022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.553249 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.553176    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prqt7\" (UniqueName: \"kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.553249 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.553236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.611038 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.611003    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:06.654020 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.653984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prqt7\" (UniqueName: \"kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.654217 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.654068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.654217 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.654198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.654368 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:20:06.654332    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-serving-cert: secret "maas-api-e2e-api-a-c59e95-serving-cert" not found
Jul 25 01:20:06.654427 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:20:06.654421    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls podName:bb9a0558-4c22-42e3-af50-cd27aec7e40d nodeName:}" failed. No retries permitted until 2026-07-25 01:20:07.154390768 +0000 UTC m=+2661.865809152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls") pod "maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" (UID: "bb9a0558-4c22-42e3-af50-cd27aec7e40d") : secret "maas-api-e2e-api-a-c59e95-serving-cert" not found
Jul 25 01:20:06.654886 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.654857    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.663255 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.663227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prqt7\" (UniqueName: \"kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:06.788446 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:06.788404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:07.158328 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.158235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:07.160793 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.160768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") pod \"maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:07.228411 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.228380    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:07.353289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.353256    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:07.420378 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.420298    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:07.581114 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.581091    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:07.584428 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:20:07.584233    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9a0558_4c22_42e3_af50_cd27aec7e40d.slice/crio-62c7268a716c1d2691062a68880aca0f19488a519b5d4d8ac306c1049341cc5c WatchSource:0}: Error finding container 62c7268a716c1d2691062a68880aca0f19488a519b5d4d8ac306c1049341cc5c: Status 404 returned error can't find the container with id 62c7268a716c1d2691062a68880aca0f19488a519b5d4d8ac306c1049341cc5c
Jul 25 01:20:07.586955 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:07.586929    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:20:08.487110 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:08.487066    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" event={"ID":"bb9a0558-4c22-42e3-af50-cd27aec7e40d","Type":"ContainerStarted","Data":"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"}
Jul 25 01:20:08.487310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:08.487130    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" event={"ID":"bb9a0558-4c22-42e3-af50-cd27aec7e40d","Type":"ContainerStarted","Data":"62c7268a716c1d2691062a68880aca0f19488a519b5d4d8ac306c1049341cc5c"}
Jul 25 01:20:08.487310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:08.487271    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:08.522336 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:08.522285    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" podStartSLOduration=1.943307951 podStartE2EDuration="2.522271469s" totalImagesPullingTime="578.963518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:08.518767891 +0000 UTC m=+2663.230186290" watchObservedRunningTime="2026-07-25 01:20:08.522271469 +0000 UTC m=+2663.233689847"
Jul 25 01:20:10.959591 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:10.959561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:11.435012 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:11.434906    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:11.816269 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:11.816239    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:12.397970 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:12.397934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:14.498914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:14.498886    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:14.780770 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:14.780677    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:14.865603 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:14.865557    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:15.043613 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:15.043502    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:15.374306 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:15.374215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:15.666217 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:15.666068    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:15.767817 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:15.767783    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:19.351504 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:19.351462    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:21.506709 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:21.506669    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 01:20:21.507127 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:21.506729    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 01:20:22.594981 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.594955    2574 generic.go:356] "Generic (PLEG): container finished" podID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerID="922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4" exitCode=0
Jul 25 01:20:22.595299 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.595008    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerDied","Data":"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"}
Jul 25 01:20:22.595417 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.595402    2574 scope.go:117] "RemoveContainer" containerID="922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"
Jul 25 01:20:22.596446 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.596417    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:22.596526 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.596495    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:22.596569 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:22.596557    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:20:23.603070 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:23.603032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerStarted","Data":"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"}
Jul 25 01:20:24.243725 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:24.243663    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:24.243981 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:24.243948    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Jul 25 01:20:24.244058 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:24.244010    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Jul 25 01:20:25.243638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:25.243589    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Jul 25 01:20:25.244640 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:25.244192    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Jul 25 01:20:26.243724 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:26.243677    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Jul 25 01:20:26.244279 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:26.243748    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Jul 25 01:20:27.243122 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.243072    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Jul 25 01:20:27.243347 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.243157    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Jul 25 01:20:27.542591 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.542476    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"]
Jul 25 01:20:27.547118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.547092    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:27.550037 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.550008    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:20:27.556234 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.556211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"]
Jul 25 01:20:27.682778 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.682739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c59e95-5fms6\" (UID: \"1510b908-d386-4660-8b32-a6cfab42fac5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:27.784240 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.784203    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c59e95-5fms6\" (UID: \"1510b908-d386-4660-8b32-a6cfab42fac5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:27.784915 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.784896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c59e95-5fms6\" (UID: \"1510b908-d386-4660-8b32-a6cfab42fac5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:27.862941 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:27.862859    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:28.015433 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.015401    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"]
Jul 25 01:20:28.016111 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:20:28.016087    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1510b908_d386_4660_8b32_a6cfab42fac5.slice/crio-19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90 WatchSource:0}: Error finding container 19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90: Status 404 returned error can't find the container with id 19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90
Jul 25 01:20:28.246850 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.246823    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:28.247312 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.247294    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:28.247928 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.247912    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:28.642835 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.642737    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6" event={"ID":"1510b908-d386-4660-8b32-a6cfab42fac5","Type":"ContainerStarted","Data":"453374ca41a694a2566da8406f1d00a310b8610c39d42b439acf46bb0d83eab9"}
Jul 25 01:20:28.642835 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.642826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6" event={"ID":"1510b908-d386-4660-8b32-a6cfab42fac5","Type":"ContainerStarted","Data":"19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90"}
Jul 25 01:20:28.662915 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:28.662868    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6" podStartSLOduration=1.662854218 podStartE2EDuration="1.662854218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:28.659732571 +0000 UTC m=+2683.371150968" watchObservedRunningTime="2026-07-25 01:20:28.662854218 +0000 UTC m=+2683.374272620"
Jul 25 01:20:29.653739 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:29.653703    2574 generic.go:356] "Generic (PLEG): container finished" podID="1510b908-d386-4660-8b32-a6cfab42fac5" containerID="453374ca41a694a2566da8406f1d00a310b8610c39d42b439acf46bb0d83eab9" exitCode=0
Jul 25 01:20:29.654251 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:29.653790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6" event={"ID":"1510b908-d386-4660-8b32-a6cfab42fac5","Type":"ContainerDied","Data":"453374ca41a694a2566da8406f1d00a310b8610c39d42b439acf46bb0d83eab9"}
Jul 25 01:20:30.792489 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:30.792467    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:30.811504 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:30.811481    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca\") pod \"1510b908-d386-4660-8b32-a6cfab42fac5\" (UID: \"1510b908-d386-4660-8b32-a6cfab42fac5\") "
Jul 25 01:20:30.811871 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:30.811844    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1510b908-d386-4660-8b32-a6cfab42fac5" (UID: "1510b908-d386-4660-8b32-a6cfab42fac5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:30.912701 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:30.912633    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1510b908-d386-4660-8b32-a6cfab42fac5-maas-api-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:31.667884 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:31.667852    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"
Jul 25 01:20:31.668054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:31.667903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6" event={"ID":"1510b908-d386-4660-8b32-a6cfab42fac5","Type":"ContainerDied","Data":"19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90"}
Jul 25 01:20:31.668054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:31.667949    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19193364eca78c2579c4c15c9bcabbb843e48096faa70b6f89c3a0ffab820d90"
Jul 25 01:20:42.712945 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.712843    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:42.713357 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.713160    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" podUID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" containerName="maas-api" containerID="cri-o://b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89" gracePeriod=30
Jul 25 01:20:42.744861 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.744832    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:42.843373 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.843343    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:42.909429 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.909399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:42.978567 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:42.978483    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:43.030289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.030256    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca\") pod \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") "
Jul 25 01:20:43.030454 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.030347    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") pod \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") "
Jul 25 01:20:43.030454 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.030428    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prqt7\" (UniqueName: \"kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7\") pod \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\" (UID: \"bb9a0558-4c22-42e3-af50-cd27aec7e40d\") "
Jul 25 01:20:43.030643 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.030620    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb9a0558-4c22-42e3-af50-cd27aec7e40d" (UID: "bb9a0558-4c22-42e3-af50-cd27aec7e40d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:43.030844 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.030828    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb9a0558-4c22-42e3-af50-cd27aec7e40d-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.032675 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.032644    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7" (OuterVolumeSpecName: "kube-api-access-prqt7") pod "bb9a0558-4c22-42e3-af50-cd27aec7e40d" (UID: "bb9a0558-4c22-42e3-af50-cd27aec7e40d"). InnerVolumeSpecName "kube-api-access-prqt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:43.032778 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.032707    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb9a0558-4c22-42e3-af50-cd27aec7e40d" (UID: "bb9a0558-4c22-42e3-af50-cd27aec7e40d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:20:43.132102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.132066    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb9a0558-4c22-42e3-af50-cd27aec7e40d-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.132102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.132098    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prqt7\" (UniqueName: \"kubernetes.io/projected/bb9a0558-4c22-42e3-af50-cd27aec7e40d-kube-api-access-prqt7\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:43.748931 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.748896    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" containerID="b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89" exitCode=0
Jul 25 01:20:43.749386 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.748936    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"
Jul 25 01:20:43.749386 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.748958    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" event={"ID":"bb9a0558-4c22-42e3-af50-cd27aec7e40d","Type":"ContainerDied","Data":"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"}
Jul 25 01:20:43.749386 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.749032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl" event={"ID":"bb9a0558-4c22-42e3-af50-cd27aec7e40d","Type":"ContainerDied","Data":"62c7268a716c1d2691062a68880aca0f19488a519b5d4d8ac306c1049341cc5c"}
Jul 25 01:20:43.749386 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.749063    2574 scope.go:117] "RemoveContainer" containerID="b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"
Jul 25 01:20:43.758819 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.758802    2574 scope.go:117] "RemoveContainer" containerID="b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"
Jul 25 01:20:43.759086 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:20:43.759065    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89\": container with ID starting with b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89 not found: ID does not exist" containerID="b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"
Jul 25 01:20:43.759127 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.759094    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89"} err="failed to get container status \"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89\": rpc error: code = NotFound desc = could not find container \"b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89\": container with ID starting with b4c5acf2b9506269b5a201d0f0d6fa9b72dae03760fbc5f8c2ed8e35ce807e89 not found: ID does not exist"
Jul 25 01:20:43.772653 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.772631    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:43.776214 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.776191    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c59e95-74cd7ff6fb-59jkl"]
Jul 25 01:20:43.934400 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:43.934371    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" path="/var/lib/kubelet/pods/bb9a0558-4c22-42e3-af50-cd27aec7e40d/volumes"
Jul 25 01:20:46.095403 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:46.095374    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:20:46.110222 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:46.110199    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:20:47.622062 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:47.622025    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"]
Jul 25 01:20:47.623711 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:47.623679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:47.624831 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:47.624810    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c59e95-5fms6"]
Jul 25 01:20:47.934052 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:47.933973    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1510b908-d386-4660-8b32-a6cfab42fac5" path="/var/lib/kubelet/pods/1510b908-d386-4660-8b32-a6cfab42fac5/volumes"
Jul 25 01:20:48.025449 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:48.025418    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:53.787346 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:53.787301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:53.907632 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:53.907590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:54.059575 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.059472    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:54.059741 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.059693    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" containerID="cri-o://d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c" gracePeriod=30
Jul 25 01:20:54.071531 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.071474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:54.324029 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.323931    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:54.439483 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.439439    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"]
Jul 25 01:20:54.440147 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440121    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1510b908-d386-4660-8b32-a6cfab42fac5" containerName="revoke-keys"
Jul 25 01:20:54.440147 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440145    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1510b908-d386-4660-8b32-a6cfab42fac5" containerName="revoke-keys"
Jul 25 01:20:54.440341 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" containerName="maas-api"
Jul 25 01:20:54.440341 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440172    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" containerName="maas-api"
Jul 25 01:20:54.440341 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440273    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb9a0558-4c22-42e3-af50-cd27aec7e40d" containerName="maas-api"
Jul 25 01:20:54.440341 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.440283    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1510b908-d386-4660-8b32-a6cfab42fac5" containerName="revoke-keys"
Jul 25 01:20:54.443420 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.443399    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:54.446055 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.446022    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:20:54.451442 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.451419    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"]
Jul 25 01:20:54.545882 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.545839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-015d51-4nd7p\" (UID: \"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:54.647118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.647023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-015d51-4nd7p\" (UID: \"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:54.647722 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.647698    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-015d51-4nd7p\" (UID: \"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:54.761754 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.761722    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:54.903828 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:54.903790    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"]
Jul 25 01:20:54.905589 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:20:54.905558    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c0d0c1_c0f8_4b61_9ab4_d62467ff16ae.slice/crio-e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b WatchSource:0}: Error finding container e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b: Status 404 returned error can't find the container with id e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b
Jul 25 01:20:55.835117 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:55.835069    2574 generic.go:356] "Generic (PLEG): container finished" podID="f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" containerID="691494d2d5acbb06f9dca5323ad1c4b34093085c01d799dc2b790d517ea6909a" exitCode=0
Jul 25 01:20:55.835332 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:55.835276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p" event={"ID":"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae","Type":"ContainerDied","Data":"691494d2d5acbb06f9dca5323ad1c4b34093085c01d799dc2b790d517ea6909a"}
Jul 25 01:20:55.835332 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:55.835325    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p" event={"ID":"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae","Type":"ContainerStarted","Data":"e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b"}
Jul 25 01:20:56.984274 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:56.984249    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:57.070663 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.070639    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca\") pod \"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae\" (UID: \"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae\") "
Jul 25 01:20:57.071008 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.070974    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" (UID: "f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:57.071578 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.071421    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae-maas-api-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:57.850244 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.850214    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"
Jul 25 01:20:57.850421 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.850259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p" event={"ID":"f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae","Type":"ContainerDied","Data":"e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b"}
Jul 25 01:20:57.850421 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:57.850315    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6b18a30a9344f5f832d4e291d347cd82abb9d8b7ce8b092e67eec0e299e4a9b"
Jul 25 01:20:58.250256 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:58.250223    2574 patch_prober.go:28] interesting pod/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 25 01:20:58.250649 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:58.250283    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 01:20:59.347998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.347972    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:59.494252 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494465 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494268    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494465 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494465 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494465 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494425    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f46ff\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494549    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494591    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494654    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy\") pod \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\" (UID: \"3eff4ea7-7ad7-4acb-b6e8-61012ff98677\") "
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494658    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494668    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:20:59.494805 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494768    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:59.495103 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.494838    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:59.495103 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.495066    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.495103 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.495095    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.495225 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.495112    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.495225 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.495126    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.495547 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.495498    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data" (OuterVolumeSpecName: "istio-data") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:59.496705 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.496669    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:20:59.496839 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.496818    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token" (OuterVolumeSpecName: "istio-token") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:59.496899 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.496871    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:59.497159 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.497139    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff" (OuterVolumeSpecName: "kube-api-access-f46ff") pod "3eff4ea7-7ad7-4acb-b6e8-61012ff98677" (UID: "3eff4ea7-7ad7-4acb-b6e8-61012ff98677"). InnerVolumeSpecName "kube-api-access-f46ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:59.595768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.595678    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.595768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.595712    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f46ff\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-kube-api-access-f46ff\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.595768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.595724    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.595768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.595733    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.595768 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.595742    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eff4ea7-7ad7-4acb-b6e8-61012ff98677-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:20:59.864374 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.864287    2574 generic.go:356] "Generic (PLEG): container finished" podID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerID="d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c" exitCode=0
Jul 25 01:20:59.864374 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.864336    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"
Jul 25 01:20:59.864610 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.864373    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerDied","Data":"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"}
Jul 25 01:20:59.864610 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.864430    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h" event={"ID":"3eff4ea7-7ad7-4acb-b6e8-61012ff98677","Type":"ContainerDied","Data":"3cf29fd750e1006f1db1451d9753a7f2cb17bbb676a4d921b2b12b304a109a70"}
Jul 25 01:20:59.864610 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.864455    2574 scope.go:117] "RemoveContainer" containerID="d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"
Jul 25 01:20:59.875018 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.874998    2574 scope.go:117] "RemoveContainer" containerID="922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"
Jul 25 01:20:59.884196 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.884172    2574 scope.go:117] "RemoveContainer" containerID="d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"
Jul 25 01:20:59.884592 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:20:59.884491    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c\": container with ID starting with d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c not found: ID does not exist" containerID="d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"
Jul 25 01:20:59.884592 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.884557    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c"} err="failed to get container status \"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c\": rpc error: code = NotFound desc = could not find container \"d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c\": container with ID starting with d81e52d7c1e53e89aaa0e159f135cdefc89f28acaf8af85e45af0f0bfb7deb8c not found: ID does not exist"
Jul 25 01:20:59.884741 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.884610    2574 scope.go:117] "RemoveContainer" containerID="922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"
Jul 25 01:20:59.884893 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:20:59.884876    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4\": container with ID starting with 922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4 not found: ID does not exist" containerID="922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"
Jul 25 01:20:59.884939 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.884900    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4"} err="failed to get container status \"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4\": rpc error: code = NotFound desc = could not find container \"922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4\": container with ID starting with 922a2a965499f80e1e4e9dbb75791f4c9d6ba3c78093a33d916852eba05086e4 not found: ID does not exist"
Jul 25 01:20:59.887233 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.887214    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:59.890976 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.890955    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c59e95-openshift-default-7d4555b4bd-jkh7h"]
Jul 25 01:20:59.933450 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:20:59.933420    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" path="/var/lib/kubelet/pods/3eff4ea7-7ad7-4acb-b6e8-61012ff98677/volumes"
Jul 25 01:21:09.957495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:09.957464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:10.025558 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:10.025524    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:10.085636 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:10.085599    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:14.508752 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:14.508718    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:14.509208 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:14.509020    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"]
Jul 25 01:21:14.511783 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:14.511758    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-015d51-4nd7p"]
Jul 25 01:21:14.917561 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:14.917471    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:15.934880 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:15.934840    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" path="/var/lib/kubelet/pods/f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae/volumes"
Jul 25 01:21:20.693396 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:20.693363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:20.754325 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:20.754296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:20.974482 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:20.974447    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:21.221007 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.220970    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:21.588440 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.588402    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:21.905975 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.905891    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:21:21.906664 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906632    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" containerName="revoke-keys"
Jul 25 01:21:21.906664 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906666    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" containerName="revoke-keys"
Jul 25 01:21:21.906803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906706    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906715    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906740    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906751    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906846    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906860    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eff4ea7-7ad7-4acb-b6e8-61012ff98677" containerName="istio-proxy"
Jul 25 01:21:21.906994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.906871    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6c0d0c1-c0f8-4b61-9ab4-d62467ff16ae" containerName="revoke-keys"
Jul 25 01:21:21.912121 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.912100    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:21.914914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.914894    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a1fe77-openshift-default-dockercfg-bxr5f\""
Jul 25 01:21:21.919827 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.919802    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:21:21.924914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:21.924877    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:22.003156 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003320 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003320 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003320 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003454 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9n4b\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003454 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003691 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003672    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.003754 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.003717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104575 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104773 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104773 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104906 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104769    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104906 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104840    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9n4b\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.104906 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104913    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104915    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.104998    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.105004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105073 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.105061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105385 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.105216    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.105425 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.105390    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.107260 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.107232    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.107482 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.107465    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.113450 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.113426    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.113684 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.113667    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9n4b\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b\") pod \"e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") " pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.206349 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.206290    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:22.226270 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.226239    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:22.379712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.379685    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:21:22.381109 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:21:22.381085    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d963b32_327b_41a7_8ea1_de7c990dd832.slice/crio-05dbe938257da7bd49b85b85404a94f58b7bf730279355198361bdb608fe6a5f WatchSource:0}: Error finding container 05dbe938257da7bd49b85b85404a94f58b7bf730279355198361bdb608fe6a5f: Status 404 returned error can't find the container with id 05dbe938257da7bd49b85b85404a94f58b7bf730279355198361bdb608fe6a5f
Jul 25 01:21:22.383263 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.383232    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:22.383332 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.383297    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:22.383376 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:22.383329    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:23.013910 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:23.013867    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerStarted","Data":"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"}
Jul 25 01:21:23.014380 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:23.013935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerStarted","Data":"05dbe938257da7bd49b85b85404a94f58b7bf730279355198361bdb608fe6a5f"}
Jul 25 01:21:23.036149 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:23.036081    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podStartSLOduration=2.036061602 podStartE2EDuration="2.036061602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:23.034907553 +0000 UTC m=+2737.746325953" watchObservedRunningTime="2026-07-25 01:21:23.036061602 +0000 UTC m=+2737.747480003"
Jul 25 01:21:23.226914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:23.226878    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:23.232078 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:23.232048    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:24.019914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:24.019881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:24.021071 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:24.021048    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:24.989946 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:24.989902    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:25.070141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.070105    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:21:25.070795 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.070620    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:25.073472 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.073454    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.075742 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.075719    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a1fe77-serving-cert\""
Jul 25 01:21:25.075860 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.075800    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9j2xk\""
Jul 25 01:21:25.084627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.084605    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:21:25.138040 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.138006    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.138259 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.138228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt6xs\" (UniqueName: \"kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.138413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.138300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.138878 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.138852    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:25.238972 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.238923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt6xs\" (UniqueName: \"kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.239171 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.238988    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.239171 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.239053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.239171 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.239087    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:25.239769 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.239742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.241673 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.241609    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.250041 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.250017    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt6xs\" (UniqueName: \"kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs\") pod \"maas-api-e2e-shared-a-a1fe77-86676f496d-48sss\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.387392 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.387357    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:25.548432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.548402    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:21:25.549956 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:21:25.549932    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70712e34_2a6a_43b5_b020_36cffa7b33b2.slice/crio-27bf40ccd30aa6ab0eba89c3526147d2e3174b09538f451ecd8f9b481934fbcb WatchSource:0}: Error finding container 27bf40ccd30aa6ab0eba89c3526147d2e3174b09538f451ecd8f9b481934fbcb: Status 404 returned error can't find the container with id 27bf40ccd30aa6ab0eba89c3526147d2e3174b09538f451ecd8f9b481934fbcb
Jul 25 01:21:25.769618 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.769588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:25.854932 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:25.854849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:26.038417 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:26.038369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" event={"ID":"70712e34-2a6a-43b5-b020-36cffa7b33b2","Type":"ContainerStarted","Data":"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"}
Jul 25 01:21:26.038619 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:26.038440    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" event={"ID":"70712e34-2a6a-43b5-b020-36cffa7b33b2","Type":"ContainerStarted","Data":"27bf40ccd30aa6ab0eba89c3526147d2e3174b09538f451ecd8f9b481934fbcb"}
Jul 25 01:21:26.038828 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:26.038793    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:26.055051 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:26.055006    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" podStartSLOduration=0.67218137 podStartE2EDuration="1.054990552s" totalImagesPullingTime="382.809182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:26.053838631 +0000 UTC m=+2740.765257031" watchObservedRunningTime="2026-07-25 01:21:26.054990552 +0000 UTC m=+2740.766408951"
Jul 25 01:21:29.534671 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:29.534638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:29.870240 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:29.870143    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:30.201145 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:30.201049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:30.340050 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:30.340010    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:32.052555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:32.052503    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:21:33.157712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.157679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.255518 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.255485    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.285730 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.285694    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:21:33.291554 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.291530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.293996 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.293970    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-d50ef4-serving-cert\""
Jul 25 01:21:33.297711 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.297687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:21:33.353743 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.353708    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.420847 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.420766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.420994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.420875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.420994 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.420909    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppg4f\" (UniqueName: \"kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.453033 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.453004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.522214 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.522179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.522393 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.522266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.522393 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.522300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppg4f\" (UniqueName: \"kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.522869 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.522846    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.524955 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.524930    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.530381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.530359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppg4f\" (UniqueName: \"kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f\") pod \"maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.607729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.607698    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:33.796849 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.796320    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:21:33.802464 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:21:33.802419    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051f2dc6_75b9_45c9_9204_2b8806416def.slice/crio-b0d1aa56e985e47a6d4f42ba4fe58bc99b4ff6ec8d74d66d3c2eff64a847f72e WatchSource:0}: Error finding container b0d1aa56e985e47a6d4f42ba4fe58bc99b4ff6ec8d74d66d3c2eff64a847f72e: Status 404 returned error can't find the container with id b0d1aa56e985e47a6d4f42ba4fe58bc99b4ff6ec8d74d66d3c2eff64a847f72e
Jul 25 01:21:33.953590 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:33.953318    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:34.053534 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:34.053362    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:34.095947 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:34.095911    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" event={"ID":"051f2dc6-75b9-45c9-9204-2b8806416def","Type":"ContainerStarted","Data":"b0d1aa56e985e47a6d4f42ba4fe58bc99b4ff6ec8d74d66d3c2eff64a847f72e"}
Jul 25 01:21:35.106074 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:35.105993    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" event={"ID":"051f2dc6-75b9-45c9-9204-2b8806416def","Type":"ContainerStarted","Data":"11cb8e8f7c889eedcdbd70ab3c3bb5ad18b3e0ca5215576540e26433aa91a1fd"}
Jul 25 01:21:35.106450 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:35.106159    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:35.120464 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:35.120398    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" podStartSLOduration=1.715461716 podStartE2EDuration="2.120382116s" totalImagesPullingTime="404.9204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:35.120128047 +0000 UTC m=+2749.831546445" watchObservedRunningTime="2026-07-25 01:21:35.120382116 +0000 UTC m=+2749.831800516"
Jul 25 01:21:37.671014 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:37.670971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:40.060669 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:40.060625    2574 patch_prober.go:28] interesting pod/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 01:21:40.061060 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:40.060679    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 01:21:41.118223 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.118192    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:21:41.157362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.157329    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerID="a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939" exitCode=0
Jul 25 01:21:41.157555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.157404    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerDied","Data":"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"}
Jul 25 01:21:41.157929 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.157910    2574 scope.go:117] "RemoveContainer" containerID="a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"
Jul 25 01:21:41.159088 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.159062    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:41.159144 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.159133    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:41.159185 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:41.159167    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:21:42.174419 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:42.174376    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerStarted","Data":"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"}
Jul 25 01:21:42.227192 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:42.227161    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:42.227360 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:42.227339    2574 patch_prober.go:28] interesting pod/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Jul 25 01:21:42.227413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:42.227394    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Jul 25 01:21:43.226655 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:43.226621    2574 patch_prober.go:28] interesting pod/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Jul 25 01:21:43.227051 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:43.226684    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Jul 25 01:21:44.226844 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:44.226795    2574 patch_prober.go:28] interesting pod/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Jul 25 01:21:44.227220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:44.226877    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Jul 25 01:21:45.227174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:45.227138    2574 patch_prober.go:28] interesting pod/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Jul 25 01:21:45.227675 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:45.227199    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Jul 25 01:21:46.231917 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:46.231886    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:46.232377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:46.232253    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:46.232976 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:46.232955    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:21:51.531584 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:21:51.531538    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:24.399033 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:22:24.398954    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:24.499155 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:22:24.499119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:25.523242 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:22:25.523209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:23:55.734295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:23:55.734191    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:41.526103 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:24:41.526063    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:42.937738 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:24:42.937696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:25:46.141054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:25:46.140976    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:25:46.158574 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:25:46.158553    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:25:46.296995 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:25:46.296959    2574 scope.go:117] "RemoveContainer" containerID="640b34f15ba0974768d72550281cb5318c6b69ca474c20da8c50c5efef647c18"
Jul 25 01:25:50.399856 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:25:50.399819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:30.636522 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:30.636480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:30.735492 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:30.735447    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:46.325395 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:46.325355    2574 scope.go:117] "RemoveContainer" containerID="453374ca41a694a2566da8406f1d00a310b8610c39d42b439acf46bb0d83eab9"
Jul 25 01:26:48.833772 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:48.833731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:49.748080 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:49.748035    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:58.612831 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:26:58.612772    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:21.597353 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:27:21.597314    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:46.352638 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:27:46.352604    2574 scope.go:117] "RemoveContainer" containerID="691494d2d5acbb06f9dca5323ad1c4b34093085c01d799dc2b790d517ea6909a"
Jul 25 01:28:24.589405 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:28:24.589323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:28:25.426367 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:28:25.426309    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:28:41.809851 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:28:41.809809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:29:07.733818 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:29:07.733771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:00.140791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.140741    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"]
Jul 25 01:30:00.144712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.144688    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"]
Jul 25 01:30:00.144849 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.144740    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"
Jul 25 01:30:00.147136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.147114    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:30:00.149141 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.149120    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"]
Jul 25 01:30:00.149943 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.149199    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"
Jul 25 01:30:00.156545 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.153801    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"]
Jul 25 01:30:00.156545 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.153896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"
Jul 25 01:30:00.156545 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.156392    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"]
Jul 25 01:30:00.161542 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.161519    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"
Jul 25 01:30:00.173046 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.173025    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"
Jul 25 01:30:00.175706 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.175672    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"]
Jul 25 01:30:00.212853 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.212834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"
Jul 25 01:30:00.346233 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.346196    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"]
Jul 25 01:30:00.347689 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:00.347643    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa468e95_1fa0_4146_bed4_5d5053f4ba7c.slice/crio-f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399 WatchSource:0}: Error finding container f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399: Status 404 returned error can't find the container with id f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399
Jul 25 01:30:00.416926 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.416896    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk" event={"ID":"aa468e95-1fa0-4146-bed4-5d5053f4ba7c","Type":"ContainerStarted","Data":"f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399"}
Jul 25 01:30:00.592999 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.592969    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"]
Jul 25 01:30:00.594355 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:00.594331    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7ab413_09ec_4195_abde_ecad9fac55e6.slice/crio-85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8 WatchSource:0}: Error finding container 85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8: Status 404 returned error can't find the container with id 85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8
Jul 25 01:30:00.596267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:00.596240    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"]
Jul 25 01:30:00.596376 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:00.596344    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb475fcda_6284_4874_a184_d4272982d24a.slice/crio-2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a WatchSource:0}: Error finding container 2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a: Status 404 returned error can't find the container with id 2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a
Jul 25 01:30:01.424454 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.424411    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk" event={"ID":"aa468e95-1fa0-4146-bed4-5d5053f4ba7c","Type":"ContainerStarted","Data":"4a596709a0bda0ae43431bcb8cfab91f4009751a86b5fbe001000f7c4c031e57"}
Jul 25 01:30:01.425938 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.425913    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr" event={"ID":"7b7ab413-09ec-4195-abde-ecad9fac55e6","Type":"ContainerStarted","Data":"32ccb8d23094ad90bf47045e90fd505feddcefcfe71083483ecd5166b072ec01"}
Jul 25 01:30:01.426069 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.425960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr" event={"ID":"7b7ab413-09ec-4195-abde-ecad9fac55e6","Type":"ContainerStarted","Data":"85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8"}
Jul 25 01:30:01.427418 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.427389    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj" event={"ID":"b475fcda-6284-4874-a184-d4272982d24a","Type":"ContainerStarted","Data":"f99fc3c11019da956cd23714cbae8acad89348b3428b07a1ad11b4b346b14c40"}
Jul 25 01:30:01.427712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.427431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj" event={"ID":"b475fcda-6284-4874-a184-d4272982d24a","Type":"ContainerStarted","Data":"2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a"}
Jul 25 01:30:01.440068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.439855    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk" podStartSLOduration=1.4398370919999999 podStartE2EDuration="1.439837092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:30:01.438999217 +0000 UTC m=+3256.150417628" watchObservedRunningTime="2026-07-25 01:30:01.439837092 +0000 UTC m=+3256.151255491"
Jul 25 01:30:01.453869 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.453822    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr" podStartSLOduration=1.453810652 podStartE2EDuration="1.453810652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:30:01.453346014 +0000 UTC m=+3256.164764415" watchObservedRunningTime="2026-07-25 01:30:01.453810652 +0000 UTC m=+3256.165229051"
Jul 25 01:30:01.469244 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:01.469206    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj" podStartSLOduration=1.469193703 podStartE2EDuration="1.469193703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:30:01.467700737 +0000 UTC m=+3256.179119134" watchObservedRunningTime="2026-07-25 01:30:01.469193703 +0000 UTC m=+3256.180612100"
Jul 25 01:30:02.435501 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.435455    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa468e95-1fa0-4146-bed4-5d5053f4ba7c" containerID="4a596709a0bda0ae43431bcb8cfab91f4009751a86b5fbe001000f7c4c031e57" exitCode=0
Jul 25 01:30:02.435975 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.435542    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk" event={"ID":"aa468e95-1fa0-4146-bed4-5d5053f4ba7c","Type":"ContainerDied","Data":"4a596709a0bda0ae43431bcb8cfab91f4009751a86b5fbe001000f7c4c031e57"}
Jul 25 01:30:02.437176 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.437152    2574 generic.go:356] "Generic (PLEG): container finished" podID="7b7ab413-09ec-4195-abde-ecad9fac55e6" containerID="32ccb8d23094ad90bf47045e90fd505feddcefcfe71083483ecd5166b072ec01" exitCode=0
Jul 25 01:30:02.437303 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.437233    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr" event={"ID":"7b7ab413-09ec-4195-abde-ecad9fac55e6","Type":"ContainerDied","Data":"32ccb8d23094ad90bf47045e90fd505feddcefcfe71083483ecd5166b072ec01"}
Jul 25 01:30:02.438685 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.438656    2574 generic.go:356] "Generic (PLEG): container finished" podID="b475fcda-6284-4874-a184-d4272982d24a" containerID="f99fc3c11019da956cd23714cbae8acad89348b3428b07a1ad11b4b346b14c40" exitCode=0
Jul 25 01:30:02.438780 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:02.438735    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj" event={"ID":"b475fcda-6284-4874-a184-d4272982d24a","Type":"ContainerDied","Data":"f99fc3c11019da956cd23714cbae8acad89348b3428b07a1ad11b4b346b14c40"}
Jul 25 01:30:03.605914 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:03.605764    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"
Jul 25 01:30:03.646553 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:03.646502    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"
Jul 25 01:30:03.672089 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:03.672058    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"
Jul 25 01:30:04.453634 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.453606    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"
Jul 25 01:30:04.453634 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.453614    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk" event={"ID":"aa468e95-1fa0-4146-bed4-5d5053f4ba7c","Type":"ContainerDied","Data":"f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399"}
Jul 25 01:30:04.453984 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.453664    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f73c124c23cb132740136ae900753edf1d6542e1173ca895f4ddb38eff3de399"
Jul 25 01:30:04.455041 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.455017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr" event={"ID":"7b7ab413-09ec-4195-abde-ecad9fac55e6","Type":"ContainerDied","Data":"85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8"}
Jul 25 01:30:04.455174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.455054    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85c4fda9b8f6922899402c3ac086a022d282b90cb278e99fd742adb0a7d892b8"
Jul 25 01:30:04.455174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.455024    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"
Jul 25 01:30:04.456447 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.456428    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj"
Jul 25 01:30:04.456586 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.456474    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749050-2bggj" event={"ID":"b475fcda-6284-4874-a184-d4272982d24a","Type":"ContainerDied","Data":"2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a"}
Jul 25 01:30:04.456586 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:04.456542    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ec0755f57e2e0a46d54d8cef139f323524eb3cadfc1c1b24384231764e4c50a"
Jul 25 01:30:21.136034 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:21.135991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:22.211729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.211690    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703336799408"]
Jul 25 01:30:22.212425 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212404    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b475fcda-6284-4874-a184-d4272982d24a" containerName="cleanup"
Jul 25 01:30:22.212477 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212428    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b475fcda-6284-4874-a184-d4272982d24a" containerName="cleanup"
Jul 25 01:30:22.212477 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212458    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa468e95-1fa0-4146-bed4-5d5053f4ba7c" containerName="cleanup"
Jul 25 01:30:22.212477 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212469    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa468e95-1fa0-4146-bed4-5d5053f4ba7c" containerName="cleanup"
Jul 25 01:30:22.212627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212492    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7ab413-09ec-4195-abde-ecad9fac55e6" containerName="cleanup"
Jul 25 01:30:22.212627 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212502    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7ab413-09ec-4195-abde-ecad9fac55e6" containerName="cleanup"
Jul 25 01:30:22.212729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212664    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b475fcda-6284-4874-a184-d4272982d24a" containerName="cleanup"
Jul 25 01:30:22.212729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212678    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa468e95-1fa0-4146-bed4-5d5053f4ba7c" containerName="cleanup"
Jul 25 01:30:22.212729 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.212689    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7ab413-09ec-4195-abde-ecad9fac55e6" containerName="cleanup"
Jul 25 01:30:22.217136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.217111    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:22.219534 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.219474    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2qsdv\""
Jul 25 01:30:22.221256 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.221229    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703336799408"]
Jul 25 01:30:22.294066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.294033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmwmc\" (UniqueName: \"kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc\") pod \"test-curl-7691-140703336799408\" (UID: \"241f8457-34fa-4c28-8324-1562aa478034\") " pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:22.395467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.395415    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmwmc\" (UniqueName: \"kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc\") pod \"test-curl-7691-140703336799408\" (UID: \"241f8457-34fa-4c28-8324-1562aa478034\") " pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:22.403328 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.403308    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmwmc\" (UniqueName: \"kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc\") pod \"test-curl-7691-140703336799408\" (UID: \"241f8457-34fa-4c28-8324-1562aa478034\") " pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:22.426895 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.426852    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:22.528925 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.528850    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:22.677266 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.677242    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703336799408"]
Jul 25 01:30:22.678894 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:22.678862    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod241f8457_34fa_4c28_8324_1562aa478034.slice/crio-87f3f303e41eac6379f8774c7fffaa977d541383e9c7b05ff118af56a34b72e6 WatchSource:0}: Error finding container 87f3f303e41eac6379f8774c7fffaa977d541383e9c7b05ff118af56a34b72e6: Status 404 returned error can't find the container with id 87f3f303e41eac6379f8774c7fffaa977d541383e9c7b05ff118af56a34b72e6
Jul 25 01:30:22.680404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:22.680380    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:30:23.582345 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:23.582310    2574 generic.go:356] "Generic (PLEG): container finished" podID="241f8457-34fa-4c28-8324-1562aa478034" containerID="be1c9f95eb040994a566c65a82880aa4a7015d53dc6e6a5c4795eb389841fece" exitCode=0
Jul 25 01:30:23.582776 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:23.582407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703336799408" event={"ID":"241f8457-34fa-4c28-8324-1562aa478034","Type":"ContainerDied","Data":"be1c9f95eb040994a566c65a82880aa4a7015d53dc6e6a5c4795eb389841fece"}
Jul 25 01:30:23.582776 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:23.582446    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703336799408" event={"ID":"241f8457-34fa-4c28-8324-1562aa478034","Type":"ContainerStarted","Data":"87f3f303e41eac6379f8774c7fffaa977d541383e9c7b05ff118af56a34b72e6"}
Jul 25 01:30:24.724541 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.724503    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:24.750256 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.750218    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703336799408_241f8457-34fa-4c28-8324-1562aa478034/test-curl-7691-140703336799408/0.log"
Jul 25 01:30:24.777068 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.776999    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703336799408"]
Jul 25 01:30:24.780320 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.780290    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703336799408"]
Jul 25 01:30:24.819573 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.819542    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmwmc\" (UniqueName: \"kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc\") pod \"241f8457-34fa-4c28-8324-1562aa478034\" (UID: \"241f8457-34fa-4c28-8324-1562aa478034\") "
Jul 25 01:30:24.821604 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.821577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc" (OuterVolumeSpecName: "kube-api-access-hmwmc") pod "241f8457-34fa-4c28-8324-1562aa478034" (UID: "241f8457-34fa-4c28-8324-1562aa478034"). InnerVolumeSpecName "kube-api-access-hmwmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:24.907651 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.907581    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703336796848"]
Jul 25 01:30:24.908056 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.908043    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f8457-34fa-4c28-8324-1562aa478034" containerName="test-curl-7691-140703336799408"
Jul 25 01:30:24.908098 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.908057    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f8457-34fa-4c28-8324-1562aa478034" containerName="test-curl-7691-140703336799408"
Jul 25 01:30:24.908157 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.908148    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f8457-34fa-4c28-8324-1562aa478034" containerName="test-curl-7691-140703336799408"
Jul 25 01:30:24.911296 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.911276    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:24.918225 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.918163    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703336796848"]
Jul 25 01:30:24.920587 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:24.920567    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmwmc\" (UniqueName: \"kubernetes.io/projected/241f8457-34fa-4c28-8324-1562aa478034-kube-api-access-hmwmc\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:25.021316 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.021286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk9fh\" (UniqueName: \"kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh\") pod \"test-curl-7691-140703336796848\" (UID: \"b82d3852-6424-430f-882f-7399dafd647c\") " pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:25.122230 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.122194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk9fh\" (UniqueName: \"kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh\") pod \"test-curl-7691-140703336796848\" (UID: \"b82d3852-6424-430f-882f-7399dafd647c\") " pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:25.130702 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.130681    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk9fh\" (UniqueName: \"kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh\") pod \"test-curl-7691-140703336796848\" (UID: \"b82d3852-6424-430f-882f-7399dafd647c\") " pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:25.222594 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.222565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:25.361140 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.361116    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703336796848"]
Jul 25 01:30:25.362862 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:25.362836    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb82d3852_6424_430f_882f_7399dafd647c.slice/crio-38f276b4f16d3695c82a1dd5fc755b0ec6e10aa84a08d9b607727c2c1fe7bea0 WatchSource:0}: Error finding container 38f276b4f16d3695c82a1dd5fc755b0ec6e10aa84a08d9b607727c2c1fe7bea0: Status 404 returned error can't find the container with id 38f276b4f16d3695c82a1dd5fc755b0ec6e10aa84a08d9b607727c2c1fe7bea0
Jul 25 01:30:25.597243 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.597155    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:25.597243 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.597170    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87f3f303e41eac6379f8774c7fffaa977d541383e9c7b05ff118af56a34b72e6"
Jul 25 01:30:25.598433 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.598406    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703336796848" event={"ID":"b82d3852-6424-430f-882f-7399dafd647c","Type":"ContainerStarted","Data":"38f276b4f16d3695c82a1dd5fc755b0ec6e10aa84a08d9b607727c2c1fe7bea0"}
Jul 25 01:30:25.607939 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:30:25.607912    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7691-140703336799408\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="241f8457-34fa-4c28-8324-1562aa478034" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:25.931745 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:30:25.931611    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7691-140703336799408\" is forbidden: User \"system:node:ip-10-0-141-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-233.ec2.internal' and this object" podUID="241f8457-34fa-4c28-8324-1562aa478034" pod="opendatahub/test-curl-7691-140703336799408"
Jul 25 01:30:25.933105 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:25.933076    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241f8457-34fa-4c28-8324-1562aa478034" path="/var/lib/kubelet/pods/241f8457-34fa-4c28-8324-1562aa478034/volumes"
Jul 25 01:30:26.605865 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:26.605829    2574 generic.go:356] "Generic (PLEG): container finished" podID="b82d3852-6424-430f-882f-7399dafd647c" containerID="5a7a6ba932585a403abdf8dea6eb577cc052b3e7b0b1c904f2dbca97bc9154a2" exitCode=0
Jul 25 01:30:26.606140 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:26.605919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703336796848" event={"ID":"b82d3852-6424-430f-882f-7399dafd647c","Type":"ContainerDied","Data":"5a7a6ba932585a403abdf8dea6eb577cc052b3e7b0b1c904f2dbca97bc9154a2"}
Jul 25 01:30:27.760095 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.760064    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:27.780462 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.780434    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703336796848_b82d3852-6424-430f-882f-7399dafd647c/test-curl-7691-140703336796848/0.log"
Jul 25 01:30:27.805841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.805817    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703336796848"]
Jul 25 01:30:27.809746 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.809724    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703336796848"]
Jul 25 01:30:27.848687 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.848662    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk9fh\" (UniqueName: \"kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh\") pod \"b82d3852-6424-430f-882f-7399dafd647c\" (UID: \"b82d3852-6424-430f-882f-7399dafd647c\") "
Jul 25 01:30:27.850674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.850649    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh" (OuterVolumeSpecName: "kube-api-access-pk9fh") pod "b82d3852-6424-430f-882f-7399dafd647c" (UID: "b82d3852-6424-430f-882f-7399dafd647c"). InnerVolumeSpecName "kube-api-access-pk9fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:27.934884 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.934803    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82d3852-6424-430f-882f-7399dafd647c" path="/var/lib/kubelet/pods/b82d3852-6424-430f-882f-7399dafd647c/volumes"
Jul 25 01:30:27.950329 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.950306    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk9fh\" (UniqueName: \"kubernetes.io/projected/b82d3852-6424-430f-882f-7399dafd647c-kube-api-access-pk9fh\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:27.976342 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.976317    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:27.976815 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.976802    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82d3852-6424-430f-882f-7399dafd647c" containerName="test-curl-7691-140703336796848"
Jul 25 01:30:27.976863 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.976817    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82d3852-6424-430f-882f-7399dafd647c" containerName="test-curl-7691-140703336796848"
Jul 25 01:30:27.976898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.976887    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82d3852-6424-430f-882f-7399dafd647c" containerName="test-curl-7691-140703336796848"
Jul 25 01:30:27.980144 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.980126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:27.985809 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:27.985783    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:28.152180 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.152146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmcg5\" (UniqueName: \"kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5\") pod \"test-curl-7691-140703333349248\" (UID: \"3b5789c6-64c1-4500-ad96-c0e4618769e9\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:28.253472 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.253438    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcg5\" (UniqueName: \"kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5\") pod \"test-curl-7691-140703333349248\" (UID: \"3b5789c6-64c1-4500-ad96-c0e4618769e9\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:28.261136 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.261110    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcg5\" (UniqueName: \"kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5\") pod \"test-curl-7691-140703333349248\" (UID: \"3b5789c6-64c1-4500-ad96-c0e4618769e9\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:28.291823 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.291795    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:28.430051 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.430022    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:28.431000 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:28.430958    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b5789c6_64c1_4500_ad96_c0e4618769e9.slice/crio-5e23a1d30a564f29ff63bce9543169561c9434080f1d010214782d58236d3aaa WatchSource:0}: Error finding container 5e23a1d30a564f29ff63bce9543169561c9434080f1d010214782d58236d3aaa: Status 404 returned error can't find the container with id 5e23a1d30a564f29ff63bce9543169561c9434080f1d010214782d58236d3aaa
Jul 25 01:30:28.620646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.620551    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703333349248" event={"ID":"3b5789c6-64c1-4500-ad96-c0e4618769e9","Type":"ContainerStarted","Data":"5e23a1d30a564f29ff63bce9543169561c9434080f1d010214782d58236d3aaa"}
Jul 25 01:30:28.621751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.621725    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703336796848"
Jul 25 01:30:28.621870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:28.621782    2574 scope.go:117] "RemoveContainer" containerID="5a7a6ba932585a403abdf8dea6eb577cc052b3e7b0b1c904f2dbca97bc9154a2"
Jul 25 01:30:29.629941 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:29.629907    2574 generic.go:356] "Generic (PLEG): container finished" podID="3b5789c6-64c1-4500-ad96-c0e4618769e9" containerID="46a1d113dcfb0d6d44df2a2e8c72308159d150a20dfa383bc5932176751513b8" exitCode=0
Jul 25 01:30:29.630432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:29.629994    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703333349248" event={"ID":"3b5789c6-64c1-4500-ad96-c0e4618769e9","Type":"ContainerDied","Data":"46a1d113dcfb0d6d44df2a2e8c72308159d150a20dfa383bc5932176751513b8"}
Jul 25 01:30:30.799519 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.799459    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:30.819706 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.819674    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703333349248_3b5789c6-64c1-4500-ad96-c0e4618769e9/test-curl-7691-140703333349248/0.log"
Jul 25 01:30:30.844480 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.844441    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:30.849373 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.849347    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:30.964267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.964231    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:30.964798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.964778    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5789c6-64c1-4500-ad96-c0e4618769e9" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:30.964798 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.964796    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5789c6-64c1-4500-ad96-c0e4618769e9" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:30.964962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.964882    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5789c6-64c1-4500-ad96-c0e4618769e9" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:30.968065 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.968047    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:30.974267 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.974211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:30.984110 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.984083    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmcg5\" (UniqueName: \"kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5\") pod \"3b5789c6-64c1-4500-ad96-c0e4618769e9\" (UID: \"3b5789c6-64c1-4500-ad96-c0e4618769e9\") "
Jul 25 01:30:30.986083 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:30.986050    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5" (OuterVolumeSpecName: "kube-api-access-fmcg5") pod "3b5789c6-64c1-4500-ad96-c0e4618769e9" (UID: "3b5789c6-64c1-4500-ad96-c0e4618769e9"). InnerVolumeSpecName "kube-api-access-fmcg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:31.085322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.085277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7pqs\" (UniqueName: \"kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs\") pod \"test-curl-7691-140703333349248\" (UID: \"8cca96c6-bb19-489b-943b-8d28131d8d61\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:31.085562 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.085373    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmcg5\" (UniqueName: \"kubernetes.io/projected/3b5789c6-64c1-4500-ad96-c0e4618769e9-kube-api-access-fmcg5\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:31.186832 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.186792    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7pqs\" (UniqueName: \"kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs\") pod \"test-curl-7691-140703333349248\" (UID: \"8cca96c6-bb19-489b-943b-8d28131d8d61\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:31.194986 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.194951    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7pqs\" (UniqueName: \"kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs\") pod \"test-curl-7691-140703333349248\" (UID: \"8cca96c6-bb19-489b-943b-8d28131d8d61\") " pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:31.280209 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.280120    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:31.418408 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.418378    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:31.419230 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:31.419206    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cca96c6_bb19_489b_943b_8d28131d8d61.slice/crio-dc88a824d10b891d1d89991a65e5a6c5ec467606a9e64a949186710df4eb5e9c WatchSource:0}: Error finding container dc88a824d10b891d1d89991a65e5a6c5ec467606a9e64a949186710df4eb5e9c: Status 404 returned error can't find the container with id dc88a824d10b891d1d89991a65e5a6c5ec467606a9e64a949186710df4eb5e9c
Jul 25 01:30:31.646922 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.646835    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703333349248" event={"ID":"8cca96c6-bb19-489b-943b-8d28131d8d61","Type":"ContainerStarted","Data":"dc88a824d10b891d1d89991a65e5a6c5ec467606a9e64a949186710df4eb5e9c"}
Jul 25 01:30:31.648124 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.648103    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:31.648242 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.648141    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e23a1d30a564f29ff63bce9543169561c9434080f1d010214782d58236d3aaa"
Jul 25 01:30:31.660422 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.660392    2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7691-140703333349248" oldPodUID="3b5789c6-64c1-4500-ad96-c0e4618769e9" podUID="8cca96c6-bb19-489b-943b-8d28131d8d61"
Jul 25 01:30:31.935596 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:31.935487    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5789c6-64c1-4500-ad96-c0e4618769e9" path="/var/lib/kubelet/pods/3b5789c6-64c1-4500-ad96-c0e4618769e9/volumes"
Jul 25 01:30:32.656102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:32.656071    2574 generic.go:356] "Generic (PLEG): container finished" podID="8cca96c6-bb19-489b-943b-8d28131d8d61" containerID="655e646033a9a2bd267ad4928bed32f486167b3de084a0cd971b5afbfc9dd05f" exitCode=0
Jul 25 01:30:32.656271 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:32.656151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703333349248" event={"ID":"8cca96c6-bb19-489b-943b-8d28131d8d61","Type":"ContainerDied","Data":"655e646033a9a2bd267ad4928bed32f486167b3de084a0cd971b5afbfc9dd05f"}
Jul 25 01:30:33.807613 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.807584    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:33.807937 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.807920    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7pqs\" (UniqueName: \"kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs\") pod \"8cca96c6-bb19-489b-943b-8d28131d8d61\" (UID: \"8cca96c6-bb19-489b-943b-8d28131d8d61\") "
Jul 25 01:30:33.809878 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.809852    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs" (OuterVolumeSpecName: "kube-api-access-g7pqs") pod "8cca96c6-bb19-489b-943b-8d28131d8d61" (UID: "8cca96c6-bb19-489b-943b-8d28131d8d61"). InnerVolumeSpecName "kube-api-access-g7pqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:33.826043 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.826020    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703333349248_8cca96c6-bb19-489b-943b-8d28131d8d61/test-curl-7691-140703333349248/0.log"
Jul 25 01:30:33.849363 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.849332    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:33.854530 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.854494    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703333349248"]
Jul 25 01:30:33.908823 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.908798    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7pqs\" (UniqueName: \"kubernetes.io/projected/8cca96c6-bb19-489b-943b-8d28131d8d61-kube-api-access-g7pqs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:33.933565 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.933488    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cca96c6-bb19-489b-943b-8d28131d8d61" path="/var/lib/kubelet/pods/8cca96c6-bb19-489b-943b-8d28131d8d61/volumes"
Jul 25 01:30:33.980667 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.980640    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:33.981111 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.981098    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cca96c6-bb19-489b-943b-8d28131d8d61" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:33.981151 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.981112    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cca96c6-bb19-489b-943b-8d28131d8d61" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:33.981205 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.981196    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cca96c6-bb19-489b-943b-8d28131d8d61" containerName="test-curl-7691-140703333349248"
Jul 25 01:30:33.984653 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.984632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:33.988263 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:33.988239    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:34.010112 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.010084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64kd\" (UniqueName: \"kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd\") pod \"test-curl-7691-140703337023072\" (UID: \"26d83076-49c7-4ed7-b7d3-63df7eb82115\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:34.111344 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.111305    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64kd\" (UniqueName: \"kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd\") pod \"test-curl-7691-140703337023072\" (UID: \"26d83076-49c7-4ed7-b7d3-63df7eb82115\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:34.119302 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.119278    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64kd\" (UniqueName: \"kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd\") pod \"test-curl-7691-140703337023072\" (UID: \"26d83076-49c7-4ed7-b7d3-63df7eb82115\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:34.296273 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.296244    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:34.433527 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.433483    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:34.433978 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:34.433947    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26d83076_49c7_4ed7_b7d3_63df7eb82115.slice/crio-f83368e9a3b133713d69f5a7e11e7daf5c5eaf33f77d24ac952cd9a9bb66be94 WatchSource:0}: Error finding container f83368e9a3b133713d69f5a7e11e7daf5c5eaf33f77d24ac952cd9a9bb66be94: Status 404 returned error can't find the container with id f83368e9a3b133713d69f5a7e11e7daf5c5eaf33f77d24ac952cd9a9bb66be94
Jul 25 01:30:34.670663 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.670577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703337023072" event={"ID":"26d83076-49c7-4ed7-b7d3-63df7eb82115","Type":"ContainerStarted","Data":"f83368e9a3b133713d69f5a7e11e7daf5c5eaf33f77d24ac952cd9a9bb66be94"}
Jul 25 01:30:34.671920 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.671897    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703333349248"
Jul 25 01:30:34.671920 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:34.671908    2574 scope.go:117] "RemoveContainer" containerID="655e646033a9a2bd267ad4928bed32f486167b3de084a0cd971b5afbfc9dd05f"
Jul 25 01:30:35.681146 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:35.681112    2574 generic.go:356] "Generic (PLEG): container finished" podID="26d83076-49c7-4ed7-b7d3-63df7eb82115" containerID="28a9c0a45eceb9b3ab2b106b08f2ed23775fef2840a443dc8eea129d57f0ec0e" exitCode=0
Jul 25 01:30:35.681506 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:35.681164    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703337023072" event={"ID":"26d83076-49c7-4ed7-b7d3-63df7eb82115","Type":"ContainerDied","Data":"28a9c0a45eceb9b3ab2b106b08f2ed23775fef2840a443dc8eea129d57f0ec0e"}
Jul 25 01:30:36.817333 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.817309    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:36.833812 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.833785    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c64kd\" (UniqueName: \"kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd\") pod \"26d83076-49c7-4ed7-b7d3-63df7eb82115\" (UID: \"26d83076-49c7-4ed7-b7d3-63df7eb82115\") "
Jul 25 01:30:36.836316 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.836292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd" (OuterVolumeSpecName: "kube-api-access-c64kd") pod "26d83076-49c7-4ed7-b7d3-63df7eb82115" (UID: "26d83076-49c7-4ed7-b7d3-63df7eb82115"). InnerVolumeSpecName "kube-api-access-c64kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:36.837054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.837023    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703337023072_26d83076-49c7-4ed7-b7d3-63df7eb82115/test-curl-7691-140703337023072/0.log"
Jul 25 01:30:36.863757 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.863727    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:36.866698 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.866675    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:36.935134 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.935106    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c64kd\" (UniqueName: \"kubernetes.io/projected/26d83076-49c7-4ed7-b7d3-63df7eb82115-kube-api-access-c64kd\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:36.995794 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.995768    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:36.996521 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.996491    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d83076-49c7-4ed7-b7d3-63df7eb82115" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:36.996648 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.996528    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d83076-49c7-4ed7-b7d3-63df7eb82115" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:36.996732 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:36.996718    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d83076-49c7-4ed7-b7d3-63df7eb82115" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:37.003707 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.003681    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:37.003833 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.003798    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.035584 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.035554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krc6d\" (UniqueName: \"kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d\") pod \"test-curl-7691-140703337023072\" (UID: \"2a691820-fffe-486d-a753-9f0176805640\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.136156 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.136121    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krc6d\" (UniqueName: \"kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d\") pod \"test-curl-7691-140703337023072\" (UID: \"2a691820-fffe-486d-a753-9f0176805640\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.143680 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.143654    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krc6d\" (UniqueName: \"kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d\") pod \"test-curl-7691-140703337023072\" (UID: \"2a691820-fffe-486d-a753-9f0176805640\") " pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.316296 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.316215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.459292 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.459265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:37.460356 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:37.460332    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a691820_fffe_486d_a753_9f0176805640.slice/crio-e594594ed1ee07d38b85aa2d7fe15a96408b0a056728d52d3fe4d44b6cedb30e WatchSource:0}: Error finding container e594594ed1ee07d38b85aa2d7fe15a96408b0a056728d52d3fe4d44b6cedb30e: Status 404 returned error can't find the container with id e594594ed1ee07d38b85aa2d7fe15a96408b0a056728d52d3fe4d44b6cedb30e
Jul 25 01:30:37.696467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.696393    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:37.696467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.696436    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f83368e9a3b133713d69f5a7e11e7daf5c5eaf33f77d24ac952cd9a9bb66be94"
Jul 25 01:30:37.697795 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.697768    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703337023072" event={"ID":"2a691820-fffe-486d-a753-9f0176805640","Type":"ContainerStarted","Data":"e594594ed1ee07d38b85aa2d7fe15a96408b0a056728d52d3fe4d44b6cedb30e"}
Jul 25 01:30:37.708444 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.708420    2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7691-140703337023072" oldPodUID="26d83076-49c7-4ed7-b7d3-63df7eb82115" podUID="2a691820-fffe-486d-a753-9f0176805640"
Jul 25 01:30:37.935282 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:37.935249    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26d83076-49c7-4ed7-b7d3-63df7eb82115" path="/var/lib/kubelet/pods/26d83076-49c7-4ed7-b7d3-63df7eb82115/volumes"
Jul 25 01:30:38.705317 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:38.705282    2574 generic.go:356] "Generic (PLEG): container finished" podID="2a691820-fffe-486d-a753-9f0176805640" containerID="3d4371c3ec0a4ec98baa4e7bba97ee8a942b2061c956ac01eec27f6a6859fb68" exitCode=0
Jul 25 01:30:38.705531 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:38.705357    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7691-140703337023072" event={"ID":"2a691820-fffe-486d-a753-9f0176805640","Type":"ContainerDied","Data":"3d4371c3ec0a4ec98baa4e7bba97ee8a942b2061c956ac01eec27f6a6859fb68"}
Jul 25 01:30:39.850867 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.850841    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:39.861490 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.861471    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krc6d\" (UniqueName: \"kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d\") pod \"2a691820-fffe-486d-a753-9f0176805640\" (UID: \"2a691820-fffe-486d-a753-9f0176805640\") "
Jul 25 01:30:39.863399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.863369    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d" (OuterVolumeSpecName: "kube-api-access-krc6d") pod "2a691820-fffe-486d-a753-9f0176805640" (UID: "2a691820-fffe-486d-a753-9f0176805640"). InnerVolumeSpecName "kube-api-access-krc6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:30:39.870324 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.870303    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7691-140703337023072_2a691820-fffe-486d-a753-9f0176805640/test-curl-7691-140703337023072/0.log"
Jul 25 01:30:39.894611 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.894588    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:39.898174 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.898138    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7691-140703337023072"]
Jul 25 01:30:39.933309 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.933278    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a691820-fffe-486d-a753-9f0176805640" path="/var/lib/kubelet/pods/2a691820-fffe-486d-a753-9f0176805640/volumes"
Jul 25 01:30:39.962940 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:39.962872    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krc6d\" (UniqueName: \"kubernetes.io/projected/2a691820-fffe-486d-a753-9f0176805640-kube-api-access-krc6d\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:30:40.434354 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:40.434252    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:40.721681 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:40.721649    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7691-140703337023072"
Jul 25 01:30:40.721834 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:40.721660    2574 scope.go:117] "RemoveContainer" containerID="3d4371c3ec0a4ec98baa4e7bba97ee8a942b2061c956ac01eec27f6a6859fb68"
Jul 25 01:30:40.752116 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:40.752085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:41.423061 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:41.423025    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:43.740696 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:43.740660    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:43.836620 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:43.836588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:43.904867 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:43.904837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:44.080731 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:44.080635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:44.494696 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:44.494639    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:44.586759 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:44.586720    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:46.185331 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:46.185299    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:30:46.204188 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:46.204165    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:30:48.237500 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:48.237462    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:59.121572 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.121468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:59.478338 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.478303    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:30:59.479076 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.479059    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a691820-fffe-486d-a753-9f0176805640" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:59.479133 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.479079    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a691820-fffe-486d-a753-9f0176805640" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:59.479206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.479195    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a691820-fffe-486d-a753-9f0176805640" containerName="test-curl-7691-140703337023072"
Jul 25 01:30:59.482573 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.482556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.486043 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.486020    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6bf6cd-openshift-default-dockercfg-twhlf\""
Jul 25 01:30:59.506107 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.506079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:30:59.524736 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.524697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:59.563054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563160    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmws4\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.563589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.563430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.608161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.608101    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:59.664248 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664216    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664248 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664499 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664499 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664499 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmws4\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.664791 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.664771    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.665195 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.665046    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.665195 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.665164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.665315 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.665244    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.667555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.667530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.667792 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.667769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.672900 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.672878    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmws4\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.673270 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.673251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token\") pod \"e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") " pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.794885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.794790    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:30:59.944797 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:30:59.944763    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53be13da_b538_46c7_adff_ed07c2fad492.slice/crio-379fb6ecf4d2f36fa2413389b119a0645d4c95c3ec8281f18b5cb6083974f4bf WatchSource:0}: Error finding container 379fb6ecf4d2f36fa2413389b119a0645d4c95c3ec8281f18b5cb6083974f4bf: Status 404 returned error can't find the container with id 379fb6ecf4d2f36fa2413389b119a0645d4c95c3ec8281f18b5cb6083974f4bf
Jul 25 01:30:59.945096 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.945068    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:30:59.947523 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.947463    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:30:59.947643 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.947620    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:30:59.947698 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:30:59.947676    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:31:00.864310 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:00.864265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerStarted","Data":"cecbebb7e7d3a332240ec41cdbbff2aaa4680dce68b2645a07dc10ccaa4291e4"}
Jul 25 01:31:00.864923 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:00.864896    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerStarted","Data":"379fb6ecf4d2f36fa2413389b119a0645d4c95c3ec8281f18b5cb6083974f4bf"}
Jul 25 01:31:00.889991 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:00.889941    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podStartSLOduration=1.889924965 podStartE2EDuration="1.889924965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:31:00.887166587 +0000 UTC m=+3315.598584996" watchObservedRunningTime="2026-07-25 01:31:00.889924965 +0000 UTC m=+3315.601343364"
Jul 25 01:31:01.795383 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:01.795346    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:01.800176 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:01.800150    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:01.870878 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:01.870848    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:01.871689 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:01.871669    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:02.832576 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.832541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:02.907946 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.907913    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:02.914301 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.914267    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:31:02.919604 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.919576    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:02.922091 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.922064    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6bf6cd-serving-cert\""
Jul 25 01:31:02.928418 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.928396    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:31:02.970981 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.970941    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:02.998280 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.998232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:02.998280 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.998274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9x8l\" (UniqueName: \"kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:02.998551 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:02.998419    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.099381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.099291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.099381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.099332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9x8l\" (UniqueName: \"kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.099632 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.099424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.099632 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:31:03.099450    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-serving-cert: secret "maas-api-e2e-ipp-b-6bf6cd-serving-cert" not found
Jul 25 01:31:03.099632 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:31:03.099555    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls podName:98f4858e-9fce-4bb1-903c-412d8dbff2db nodeName:}" failed. No retries permitted until 2026-07-25 01:31:03.599505691 +0000 UTC m=+3318.310924070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls") pod "maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" (UID: "98f4858e-9fce-4bb1-903c-412d8dbff2db") : secret "maas-api-e2e-ipp-b-6bf6cd-serving-cert" not found
Jul 25 01:31:03.100183 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.100162    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.110784 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.110754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9x8l\" (UniqueName: \"kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.190206 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.190163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:03.484674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.484640    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:03.582965 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.582928    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:03.605634 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.605595    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.608428 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.608398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:03.838429 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:03.838331    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:04.001946 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:04.001915    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:31:04.003152 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:31:04.003123    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f4858e_9fce_4bb1_903c_412d8dbff2db.slice/crio-455977ebf5e6d435193ef45e728527e0bef5f881e503d673e3891aaaa0a8e88f WatchSource:0}: Error finding container 455977ebf5e6d435193ef45e728527e0bef5f881e503d673e3891aaaa0a8e88f: Status 404 returned error can't find the container with id 455977ebf5e6d435193ef45e728527e0bef5f881e503d673e3891aaaa0a8e88f
Jul 25 01:31:04.893586 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:04.893549    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" event={"ID":"98f4858e-9fce-4bb1-903c-412d8dbff2db","Type":"ContainerStarted","Data":"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"}
Jul 25 01:31:04.893744 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:04.893614    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" event={"ID":"98f4858e-9fce-4bb1-903c-412d8dbff2db","Type":"ContainerStarted","Data":"455977ebf5e6d435193ef45e728527e0bef5f881e503d673e3891aaaa0a8e88f"}
Jul 25 01:31:04.893744 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:04.893645    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:04.910098 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:04.910048    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" podStartSLOduration=2.581378203 podStartE2EDuration="2.910032315s" totalImagesPullingTime="328.654112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:31:04.908296121 +0000 UTC m=+3319.619714519" watchObservedRunningTime="2026-07-25 01:31:04.910032315 +0000 UTC m=+3319.621450713"
Jul 25 01:31:07.342578 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:07.342535    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:10.906091 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:10.906009    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:31:17.912700 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:17.912655    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 01:31:17.913161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:17.912721    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 01:31:19.001833 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.001802    2574 generic.go:356] "Generic (PLEG): container finished" podID="53be13da-b538-46c7-adff-ed07c2fad492" containerID="cecbebb7e7d3a332240ec41cdbbff2aaa4680dce68b2645a07dc10ccaa4291e4" exitCode=0
Jul 25 01:31:19.002273 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.001872    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerDied","Data":"cecbebb7e7d3a332240ec41cdbbff2aaa4680dce68b2645a07dc10ccaa4291e4"}
Jul 25 01:31:19.002273 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.002214    2574 scope.go:117] "RemoveContainer" containerID="cecbebb7e7d3a332240ec41cdbbff2aaa4680dce68b2645a07dc10ccaa4291e4"
Jul 25 01:31:19.003289 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.003262    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:31:19.003345 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.003328    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:31:19.003385 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.003360    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 01:31:19.630539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:19.630469    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:31:20.018926 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:20.018894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerStarted","Data":"a6d16b9220dc037daa2171ed19daf888cd587108809b7c90fb02a9b6b8b0127a"}
Jul 25 01:31:20.795376 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:20.795331    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:20.795632 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:20.795605    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 25 01:31:20.795736 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:20.795668    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 25 01:31:21.796170 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:21.796133    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 25 01:31:21.796594 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:21.796199    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 25 01:31:22.795751 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:22.795716    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 25 01:31:22.795941 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:22.795784    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 25 01:31:23.795834 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:23.795798    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Jul 25 01:31:23.796236 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:23.795863    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Jul 25 01:31:24.546920 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:24.546884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:31:24.798698 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:24.798611    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:24.799142 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:24.799080    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:24.799697 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:24.799678    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:31:26.609893 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:31:26.609853    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:06.793087 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:06.793047    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:32:07.289467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:07.289427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:07.652827 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:07.652728    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:08.021106 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:08.021071    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:08.384108 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:08.384004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.118182 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.118145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.202328 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.202293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.202930 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.202908    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:16.207029 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.207012    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.209404 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.209381    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-03dd34-serving-cert\""
Jul 25 01:32:16.217322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.217298    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:16.275090 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.275054    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.282979 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.282941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxn4\" (UniqueName: \"kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.283205 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.283047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.283205 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.283140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.384425 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.384331    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.384425 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.384409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.384670 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.384495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxn4\" (UniqueName: \"kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.384670 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:32:16.384587    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-serving-cert: secret "maas-api-e2e-ipp-cleanup-03dd34-serving-cert" not found
Jul 25 01:32:16.384670 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:32:16.384654    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls podName:43c48a75-b491-436e-be3b-16963c55c262 nodeName:}" failed. No retries permitted until 2026-07-25 01:32:16.884637881 +0000 UTC m=+3391.596056262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" (UID: "43c48a75-b491-436e-be3b-16963c55c262") : secret "maas-api-e2e-ipp-cleanup-03dd34-serving-cert" not found
Jul 25 01:32:16.385096 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.385076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.397161 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.397121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxn4\" (UniqueName: \"kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.438465 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.438426    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.883227 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.883178    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:16.888378 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.888336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.891028 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.890994    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:16.976210 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:16.976167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:17.121830 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:17.121742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:17.289288 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:17.289258    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:17.290045 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:32:17.290017    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c48a75_b491_436e_be3b_16963c55c262.slice/crio-51e760a6c4cb01fe4def39a15bec3b4e68728667d46cf494a4a5d5139f2f5346 WatchSource:0}: Error finding container 51e760a6c4cb01fe4def39a15bec3b4e68728667d46cf494a4a5d5139f2f5346: Status 404 returned error can't find the container with id 51e760a6c4cb01fe4def39a15bec3b4e68728667d46cf494a4a5d5139f2f5346
Jul 25 01:32:17.426955 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:17.426860    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" event={"ID":"43c48a75-b491-436e-be3b-16963c55c262","Type":"ContainerStarted","Data":"51e760a6c4cb01fe4def39a15bec3b4e68728667d46cf494a4a5d5139f2f5346"}
Jul 25 01:32:18.438006 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:18.437951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" event={"ID":"43c48a75-b491-436e-be3b-16963c55c262","Type":"ContainerStarted","Data":"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"}
Jul 25 01:32:18.438546 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:18.438093    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:18.460278 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:18.460223    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" podStartSLOduration=1.966644877 podStartE2EDuration="2.460203394s" totalImagesPullingTime="493.558517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:32:18.459453161 +0000 UTC m=+3393.170871556" watchObservedRunningTime="2026-07-25 01:32:18.460203394 +0000 UTC m=+3393.171621793"
Jul 25 01:32:20.652959 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:20.652906    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:24.450737 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:24.450700    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:28.755589 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:28.755541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:28.854928 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:28.854889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:31.627408 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.627367    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"]
Jul 25 01:32:31.631466 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.631447    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:31.634152 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.634127    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhxxg\""
Jul 25 01:32:31.638131 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.638103    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"]
Jul 25 01:32:31.741732 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.741690    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72\" (UID: \"0898b8de-0624-434f-ba66-ac2c5bc2d905\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:31.843313 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.843273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72\" (UID: \"0898b8de-0624-434f-ba66-ac2c5bc2d905\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:31.844080 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.844045    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72\" (UID: \"0898b8de-0624-434f-ba66-ac2c5bc2d905\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:31.949208 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:31.949126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:32.098415 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:32.098385    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"]
Jul 25 01:32:32.100202 ip-10-0-141-233 kubenswrapper[2574]: W0725 01:32:32.100173    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0898b8de_0624_434f_ba66_ac2c5bc2d905.slice/crio-5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065 WatchSource:0}: Error finding container 5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065: Status 404 returned error can't find the container with id 5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065
Jul 25 01:32:32.536130 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:32.536086    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72" event={"ID":"0898b8de-0624-434f-ba66-ac2c5bc2d905","Type":"ContainerStarted","Data":"92bebf2c14afffb64465ddbbe0f045463c35d38a9dce758c05b8b8c405401e9c"}
Jul 25 01:32:32.536602 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:32.536159    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72" event={"ID":"0898b8de-0624-434f-ba66-ac2c5bc2d905","Type":"ContainerStarted","Data":"5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065"}
Jul 25 01:32:32.554147 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:32.554099    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72" podStartSLOduration=1.554081508 podStartE2EDuration="1.554081508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:32:32.552739352 +0000 UTC m=+3407.264157750" watchObservedRunningTime="2026-07-25 01:32:32.554081508 +0000 UTC m=+3407.265499906"
Jul 25 01:32:33.543923 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:33.543882    2574 generic.go:356] "Generic (PLEG): container finished" podID="0898b8de-0624-434f-ba66-ac2c5bc2d905" containerID="92bebf2c14afffb64465ddbbe0f045463c35d38a9dce758c05b8b8c405401e9c" exitCode=0
Jul 25 01:32:33.544333 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:33.543967    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72" event={"ID":"0898b8de-0624-434f-ba66-ac2c5bc2d905","Type":"ContainerDied","Data":"92bebf2c14afffb64465ddbbe0f045463c35d38a9dce758c05b8b8c405401e9c"}
Jul 25 01:32:34.688555 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:34.688532    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:34.772362 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:34.772328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca\") pod \"0898b8de-0624-434f-ba66-ac2c5bc2d905\" (UID: \"0898b8de-0624-434f-ba66-ac2c5bc2d905\") "
Jul 25 01:32:34.772702 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:34.772675    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0898b8de-0624-434f-ba66-ac2c5bc2d905" (UID: "0898b8de-0624-434f-ba66-ac2c5bc2d905"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:32:34.873319 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:34.873233    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0898b8de-0624-434f-ba66-ac2c5bc2d905-maas-api-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:32:35.558796 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:35.558764    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"
Jul 25 01:32:35.558796 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:35.558771    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72" event={"ID":"0898b8de-0624-434f-ba66-ac2c5bc2d905","Type":"ContainerDied","Data":"5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065"}
Jul 25 01:32:35.559015 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:35.558816    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e3354ead5c903c7475eb30830f001f38766d492a5dfe4d9b9150a3eff682065"
Jul 25 01:32:38.015704 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:38.015661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:42.182059 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.182021    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:42.182920 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.182848    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" podUID="43c48a75-b491-436e-be3b-16963c55c262" containerName="maas-api" containerID="cri-o://6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9" gracePeriod=30
Jul 25 01:32:42.198025 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.197975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:42.269278 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.269243    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:42.331029 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.330997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:42.453809 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.453782    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:42.551851 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.551814    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpxn4\" (UniqueName: \"kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4\") pod \"43c48a75-b491-436e-be3b-16963c55c262\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") "
Jul 25 01:32:42.552055 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.551887    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca\") pod \"43c48a75-b491-436e-be3b-16963c55c262\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") "
Jul 25 01:32:42.552055 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.552010    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") pod \"43c48a75-b491-436e-be3b-16963c55c262\" (UID: \"43c48a75-b491-436e-be3b-16963c55c262\") "
Jul 25 01:32:42.552611 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.552573    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43c48a75-b491-436e-be3b-16963c55c262" (UID: "43c48a75-b491-436e-be3b-16963c55c262"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:32:42.554137 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.554104    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4" (OuterVolumeSpecName: "kube-api-access-rpxn4") pod "43c48a75-b491-436e-be3b-16963c55c262" (UID: "43c48a75-b491-436e-be3b-16963c55c262"). InnerVolumeSpecName "kube-api-access-rpxn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:32:42.554277 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.554252    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43c48a75-b491-436e-be3b-16963c55c262" (UID: "43c48a75-b491-436e-be3b-16963c55c262"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:32:42.607877 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.607838    2574 generic.go:356] "Generic (PLEG): container finished" podID="43c48a75-b491-436e-be3b-16963c55c262" containerID="6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9" exitCode=0
Jul 25 01:32:42.608032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.607885    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"
Jul 25 01:32:42.608032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.607899    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" event={"ID":"43c48a75-b491-436e-be3b-16963c55c262","Type":"ContainerDied","Data":"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"}
Jul 25 01:32:42.608032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.607950    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx" event={"ID":"43c48a75-b491-436e-be3b-16963c55c262","Type":"ContainerDied","Data":"51e760a6c4cb01fe4def39a15bec3b4e68728667d46cf494a4a5d5139f2f5346"}
Jul 25 01:32:42.608032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.607974    2574 scope.go:117] "RemoveContainer" containerID="6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"
Jul 25 01:32:42.619378 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.619361    2574 scope.go:117] "RemoveContainer" containerID="6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"
Jul 25 01:32:42.619666 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:32:42.619647    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9\": container with ID starting with 6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9 not found: ID does not exist" containerID="6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"
Jul 25 01:32:42.619716 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.619676    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9"} err="failed to get container status \"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9\": rpc error: code = NotFound desc = could not find container \"6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9\": container with ID starting with 6a3cabfff14137832ca009f0ce07f02da42b9ea270b2458f90b3c76a84f9b3d9 not found: ID does not exist"
Jul 25 01:32:42.633562 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.633387    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:42.637083 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.637060    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-03dd34-586f4464c8-clfgx"]
Jul 25 01:32:42.653389 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.653363    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c48a75-b491-436e-be3b-16963c55c262-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:32:42.653389 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.653388    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c48a75-b491-436e-be3b-16963c55c262-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:32:42.653535 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:42.653398    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpxn4\" (UniqueName: \"kubernetes.io/projected/43c48a75-b491-436e-be3b-16963c55c262-kube-api-access-rpxn4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:32:43.934167 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:43.934127    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c48a75-b491-436e-be3b-16963c55c262" path="/var/lib/kubelet/pods/43c48a75-b491-436e-be3b-16963c55c262/volumes"
Jul 25 01:32:46.694288 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:46.694201    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"]
Jul 25 01:32:46.697162 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:46.697141    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-03dd34-r7r72"]
Jul 25 01:32:46.700727 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:46.700703    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:47.095870 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:47.095837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:47.935531 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:47.935475    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0898b8de-0624-434f-ba66-ac2c5bc2d905" path="/var/lib/kubelet/pods/0898b8de-0624-434f-ba66-ac2c5bc2d905/volumes"
Jul 25 01:32:53.020168 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:53.020133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:53.077970 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:53.077927    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:53.306823 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:53.306721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:53.581804 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:32:53.581716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:12.718865 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:12.718826    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:12.792789 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:12.792747    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:12.917489 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:12.917456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:17.306782 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:17.306741    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:17.697200 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:17.697101    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:22.832874 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:22.832833    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:58.924459 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:58.924423    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:33:59.816447 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:33:59.816412    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:05.217397 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:05.217363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:05.275445 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:05.275414    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:05.508748 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:05.508656    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:05.778862 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:05.778761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:16.374213 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.374122    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:34:16.374979 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.374942    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" podUID="98f4858e-9fce-4bb1-903c-412d8dbff2db" containerName="maas-api" containerID="cri-o://b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371" gracePeriod=30
Jul 25 01:34:16.394639 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.394607    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:16.465459 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.465421    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:16.526746 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.526713    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:16.645117 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.645092    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:34:16.816345 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.816305    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca\") pod \"98f4858e-9fce-4bb1-903c-412d8dbff2db\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") "
Jul 25 01:34:16.816543 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.816376    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") pod \"98f4858e-9fce-4bb1-903c-412d8dbff2db\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") "
Jul 25 01:34:16.816543 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.816395    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9x8l\" (UniqueName: \"kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l\") pod \"98f4858e-9fce-4bb1-903c-412d8dbff2db\" (UID: \"98f4858e-9fce-4bb1-903c-412d8dbff2db\") "
Jul 25 01:34:16.816686 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.816663    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98f4858e-9fce-4bb1-903c-412d8dbff2db" (UID: "98f4858e-9fce-4bb1-903c-412d8dbff2db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:16.818384 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.818359    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98f4858e-9fce-4bb1-903c-412d8dbff2db" (UID: "98f4858e-9fce-4bb1-903c-412d8dbff2db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:34:16.818491 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.818428    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l" (OuterVolumeSpecName: "kube-api-access-n9x8l") pod "98f4858e-9fce-4bb1-903c-412d8dbff2db" (UID: "98f4858e-9fce-4bb1-903c-412d8dbff2db"). InnerVolumeSpecName "kube-api-access-n9x8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:16.918066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.917992    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98f4858e-9fce-4bb1-903c-412d8dbff2db-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:16.918066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.918028    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9x8l\" (UniqueName: \"kubernetes.io/projected/98f4858e-9fce-4bb1-903c-412d8dbff2db-kube-api-access-n9x8l\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:16.918066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:16.918038    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f4858e-9fce-4bb1-903c-412d8dbff2db-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:17.265468 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.265431    2574 generic.go:356] "Generic (PLEG): container finished" podID="98f4858e-9fce-4bb1-903c-412d8dbff2db" containerID="b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371" exitCode=0
Jul 25 01:34:17.265666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.265471    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"
Jul 25 01:34:17.265666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.265533    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" event={"ID":"98f4858e-9fce-4bb1-903c-412d8dbff2db","Type":"ContainerDied","Data":"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"}
Jul 25 01:34:17.265666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.265587    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg" event={"ID":"98f4858e-9fce-4bb1-903c-412d8dbff2db","Type":"ContainerDied","Data":"455977ebf5e6d435193ef45e728527e0bef5f881e503d673e3891aaaa0a8e88f"}
Jul 25 01:34:17.265666 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.265614    2574 scope.go:117] "RemoveContainer" containerID="b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"
Jul 25 01:34:17.276225 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.276208    2574 scope.go:117] "RemoveContainer" containerID="b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"
Jul 25 01:34:17.276505 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:34:17.276459    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371\": container with ID starting with b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371 not found: ID does not exist" containerID="b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"
Jul 25 01:34:17.276505 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.276490    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371"} err="failed to get container status \"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371\": rpc error: code = NotFound desc = could not find container \"b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371\": container with ID starting with b2c82f432054955efb5de43b446974e5b44ac2bdcdf6340214f52f6ee3a00371 not found: ID does not exist"
Jul 25 01:34:17.287330 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.287306    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:34:17.291471 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.291448    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6bf6cd-6555f85fbf-sppvg"]
Jul 25 01:34:17.933796 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:17.933761    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f4858e-9fce-4bb1-903c-412d8dbff2db" path="/var/lib/kubelet/pods/98f4858e-9fce-4bb1-903c-412d8dbff2db/volumes"
Jul 25 01:34:18.390425 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:18.390388    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:20.983743 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:20.983702    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:21.401047 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:21.400955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:26.960045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:26.960007    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:27.020704 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:27.020669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:27.235005 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:27.234918    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:34:27.235208 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:27.235183    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy" containerID="cri-o://a6d16b9220dc037daa2171ed19daf888cd587108809b7c90fb02a9b6b8b0127a" gracePeriod=30
Jul 25 01:34:27.243778 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:27.243744    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:27.497030 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:27.496948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:32.382221 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.382176    2574 generic.go:356] "Generic (PLEG): container finished" podID="53be13da-b538-46c7-adff-ed07c2fad492" containerID="a6d16b9220dc037daa2171ed19daf888cd587108809b7c90fb02a9b6b8b0127a" exitCode=0
Jul 25 01:34:32.382682 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.382262    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerDied","Data":"a6d16b9220dc037daa2171ed19daf888cd587108809b7c90fb02a9b6b8b0127a"}
Jul 25 01:34:32.382682 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.382342    2574 scope.go:117] "RemoveContainer" containerID="cecbebb7e7d3a332240ec41cdbbff2aaa4680dce68b2645a07dc10ccaa4291e4"
Jul 25 01:34:32.508502 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.508442    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:34:32.563187 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563156    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563199    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563218    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563235    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmws4\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563280    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563350 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563318    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563524    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563646 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563620    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs\") pod \"53be13da-b538-46c7-adff-ed07c2fad492\" (UID: \"53be13da-b538-46c7-adff-ed07c2fad492\") "
Jul 25 01:34:32.563827 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563698    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:32.563885 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.563813    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:32.564041 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.564022    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.564118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.564031    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data" (OuterVolumeSpecName: "istio-data") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:32.564118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.564045    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.564118 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.564101    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:32.564291 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.564146    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:32.565549 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.565494    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4" (OuterVolumeSpecName: "kube-api-access-mmws4") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "kube-api-access-mmws4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:32.565549 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.565527    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:32.565778 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.565759    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:34:32.565892 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.565878    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token" (OuterVolumeSpecName: "istio-token") pod "53be13da-b538-46c7-adff-ed07c2fad492" (UID: "53be13da-b538-46c7-adff-ed07c2fad492"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:32.665295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665258    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665288    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665298    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53be13da-b538-46c7-adff-ed07c2fad492-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665295 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665306    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53be13da-b538-46c7-adff-ed07c2fad492-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665603 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665318    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmws4\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-kube-api-access-mmws4\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665603 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665327    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53be13da-b538-46c7-adff-ed07c2fad492-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:32.665603 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:32.665334    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53be13da-b538-46c7-adff-ed07c2fad492-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:33.390248 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.390214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4" event={"ID":"53be13da-b538-46c7-adff-ed07c2fad492","Type":"ContainerDied","Data":"379fb6ecf4d2f36fa2413389b119a0645d4c95c3ec8281f18b5cb6083974f4bf"}
Jul 25 01:34:33.390248 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.390241    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"
Jul 25 01:34:33.390777 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.390282    2574 scope.go:117] "RemoveContainer" containerID="a6d16b9220dc037daa2171ed19daf888cd587108809b7c90fb02a9b6b8b0127a"
Jul 25 01:34:33.421826 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.421800    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:34:33.425197 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.425175    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6bf6cd-openshift-default-5f987c69f4-h9zl4"]
Jul 25 01:34:33.933908 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:33.933871    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53be13da-b538-46c7-adff-ed07c2fad492" path="/var/lib/kubelet/pods/53be13da-b538-46c7-adff-ed07c2fad492/volumes"
Jul 25 01:34:37.995327 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:37.995290    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:34:37.995823 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:37.995535    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" podUID="70712e34-2a6a-43b5-b020-36cffa7b33b2" containerName="maas-api" containerID="cri-o://98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80" gracePeriod=30
Jul 25 01:34:38.014061 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.014036    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"]
Jul 25 01:34:38.017270 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.017243    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a1fe77-29749050-6msnk"]
Jul 25 01:34:38.018283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.018258    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:38.087496 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.087461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:38.140493 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.140450    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:38.260520 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.260441    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:34:38.319560 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.319532    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls\") pod \"70712e34-2a6a-43b5-b020-36cffa7b33b2\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") "
Jul 25 01:34:38.319708 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.319590    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca\") pod \"70712e34-2a6a-43b5-b020-36cffa7b33b2\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") "
Jul 25 01:34:38.319708 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.319631    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt6xs\" (UniqueName: \"kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs\") pod \"70712e34-2a6a-43b5-b020-36cffa7b33b2\" (UID: \"70712e34-2a6a-43b5-b020-36cffa7b33b2\") "
Jul 25 01:34:38.319942 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.319916    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70712e34-2a6a-43b5-b020-36cffa7b33b2" (UID: "70712e34-2a6a-43b5-b020-36cffa7b33b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:38.321551 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.321505    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70712e34-2a6a-43b5-b020-36cffa7b33b2" (UID: "70712e34-2a6a-43b5-b020-36cffa7b33b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:34:38.321650 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.321626    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs" (OuterVolumeSpecName: "kube-api-access-jt6xs") pod "70712e34-2a6a-43b5-b020-36cffa7b33b2" (UID: "70712e34-2a6a-43b5-b020-36cffa7b33b2"). InnerVolumeSpecName "kube-api-access-jt6xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:38.420692 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.420667    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70712e34-2a6a-43b5-b020-36cffa7b33b2-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.420692 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.420692    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70712e34-2a6a-43b5-b020-36cffa7b33b2-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.420862 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.420701    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt6xs\" (UniqueName: \"kubernetes.io/projected/70712e34-2a6a-43b5-b020-36cffa7b33b2-kube-api-access-jt6xs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:38.423397 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.423364    2574 generic.go:356] "Generic (PLEG): container finished" podID="70712e34-2a6a-43b5-b020-36cffa7b33b2" containerID="98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80" exitCode=0
Jul 25 01:34:38.423495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.423411    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"
Jul 25 01:34:38.423495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.423451    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" event={"ID":"70712e34-2a6a-43b5-b020-36cffa7b33b2","Type":"ContainerDied","Data":"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"}
Jul 25 01:34:38.423608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.423520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss" event={"ID":"70712e34-2a6a-43b5-b020-36cffa7b33b2","Type":"ContainerDied","Data":"27bf40ccd30aa6ab0eba89c3526147d2e3174b09538f451ecd8f9b481934fbcb"}
Jul 25 01:34:38.423608 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.423531    2574 scope.go:117] "RemoveContainer" containerID="98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"
Jul 25 01:34:38.434247 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.434227    2574 scope.go:117] "RemoveContainer" containerID="98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"
Jul 25 01:34:38.434544 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:34:38.434519    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80\": container with ID starting with 98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80 not found: ID does not exist" containerID="98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"
Jul 25 01:34:38.434619 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.434556    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80"} err="failed to get container status \"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80\": rpc error: code = NotFound desc = could not find container \"98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80\": container with ID starting with 98f218134e6e12a9fae41f71e49da0c742102254a9e9b3d61b73b602285d2d80 not found: ID does not exist"
Jul 25 01:34:38.447150 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.447128    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:34:38.453376 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:38.453343    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a1fe77-86676f496d-48sss"]
Jul 25 01:34:39.934334 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:39.934301    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70712e34-2a6a-43b5-b020-36cffa7b33b2" path="/var/lib/kubelet/pods/70712e34-2a6a-43b5-b020-36cffa7b33b2/volumes"
Jul 25 01:34:39.934726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:39.934611    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa468e95-1fa0-4146-bed4-5d5053f4ba7c" path="/var/lib/kubelet/pods/aa468e95-1fa0-4146-bed4-5d5053f4ba7c/volumes"
Jul 25 01:34:42.709901 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:42.709865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:43.131458 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:43.131371    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:48.932784 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:48.932746    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:49.032997 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:49.032962    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:49.243899 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:49.243870    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:34:49.244110 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:49.244082    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy" containerID="cri-o://223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4" gracePeriod=30
Jul 25 01:34:49.264528 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:49.264486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:49.546495 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:49.546398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:54.510300 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.510235    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:34:54.536072 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.536036    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerID="223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4" exitCode=0
Jul 25 01:34:54.536226 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.536080    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"
Jul 25 01:34:54.536226 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.536083    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerDied","Data":"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"}
Jul 25 01:34:54.536226 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.536216    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx" event={"ID":"2d963b32-327b-41a7-8ea1-de7c990dd832","Type":"ContainerDied","Data":"05dbe938257da7bd49b85b85404a94f58b7bf730279355198361bdb608fe6a5f"}
Jul 25 01:34:54.536390 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.536250    2574 scope.go:117] "RemoveContainer" containerID="223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"
Jul 25 01:34:54.546326 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.546308    2574 scope.go:117] "RemoveContainer" containerID="a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"
Jul 25 01:34:54.555322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.555304    2574 scope.go:117] "RemoveContainer" containerID="223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"
Jul 25 01:34:54.555662 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:34:54.555636    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4\": container with ID starting with 223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4 not found: ID does not exist" containerID="223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"
Jul 25 01:34:54.555733 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.555675    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4"} err="failed to get container status \"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4\": rpc error: code = NotFound desc = could not find container \"223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4\": container with ID starting with 223c4d830669503302f1cdc583e000f92e2a579b8a3659d490dc9e6583ec46d4 not found: ID does not exist"
Jul 25 01:34:54.555733 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.555700    2574 scope.go:117] "RemoveContainer" containerID="a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"
Jul 25 01:34:54.555947 ip-10-0-141-233 kubenswrapper[2574]: E0725 01:34:54.555926    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939\": container with ID starting with a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939 not found: ID does not exist" containerID="a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"
Jul 25 01:34:54.555989 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.555952    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939"} err="failed to get container status \"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939\": rpc error: code = NotFound desc = could not find container \"a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939\": container with ID starting with a9df73c7b58eb8b122c5408a318a4299a6b2b8e3b2ace68e6aba617124f16939 not found: ID does not exist"
Jul 25 01:34:54.676793 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.676760    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.676956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.676799    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.676956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.676832    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.676956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.676859    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9n4b\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.676956 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.676930    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677036    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677069    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677133    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677167    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs\") pod \"2d963b32-327b-41a7-8ea1-de7c990dd832\" (UID: \"2d963b32-327b-41a7-8ea1-de7c990dd832\") "
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677229    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677558    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677645    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677774    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677798    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:54.677998 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.677860    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:54.679631 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.679595    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:34:54.679728 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.679624    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:54.679728 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.679651    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b" (OuterVolumeSpecName: "kube-api-access-r9n4b") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "kube-api-access-r9n4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:34:54.679728 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.679673    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d963b32-327b-41a7-8ea1-de7c990dd832" (UID: "2d963b32-327b-41a7-8ea1-de7c990dd832"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778808    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-data\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778837    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-token\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778847    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9n4b\" (UniqueName: \"kubernetes.io/projected/2d963b32-327b-41a7-8ea1-de7c990dd832-kube-api-access-r9n4b\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778856    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-credential-socket\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778865    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-envoy\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778873    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d963b32-327b-41a7-8ea1-de7c990dd832-istio-podinfo\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778886    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d963b32-327b-41a7-8ea1-de7c990dd832-istiod-ca-cert\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.778898 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.778898    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d963b32-327b-41a7-8ea1-de7c990dd832-workload-certs\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:34:54.859491 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.859464    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:34:54.863381 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:54.863357    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a1fe77-openshift-default-75fdd9b8c6-gktmx"]
Jul 25 01:34:55.933957 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:34:55.933924    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" path="/var/lib/kubelet/pods/2d963b32-327b-41a7-8ea1-de7c990dd832/volumes"
Jul 25 01:35:00.404111 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.404072    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:35:00.404622 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.404335    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" podUID="051f2dc6-75b9-45c9-9204-2b8806416def" containerName="maas-api" containerID="cri-o://11cb8e8f7c889eedcdbd70ab3c3bb5ad18b3e0ca5215576540e26433aa91a1fd" gracePeriod=30
Jul 25 01:35:00.436162 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.436134    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"]
Jul 25 01:35:00.440050 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.440021    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d50ef4-29749050-b4sqr"]
Jul 25 01:35:00.444186 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.444160    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:00.507339 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.507300    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:00.578101 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.578065    2574 generic.go:356] "Generic (PLEG): container finished" podID="051f2dc6-75b9-45c9-9204-2b8806416def" containerID="11cb8e8f7c889eedcdbd70ab3c3bb5ad18b3e0ca5215576540e26433aa91a1fd" exitCode=0
Jul 25 01:35:00.578346 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.578293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" event={"ID":"051f2dc6-75b9-45c9-9204-2b8806416def","Type":"ContainerDied","Data":"11cb8e8f7c889eedcdbd70ab3c3bb5ad18b3e0ca5215576540e26433aa91a1fd"}
Jul 25 01:35:00.607937 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.607897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:00.668439 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.668380    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:35:00.835220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.835180    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppg4f\" (UniqueName: \"kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f\") pod \"051f2dc6-75b9-45c9-9204-2b8806416def\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") "
Jul 25 01:35:00.835391 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.835234    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls\") pod \"051f2dc6-75b9-45c9-9204-2b8806416def\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") "
Jul 25 01:35:00.835391 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.835281    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca\") pod \"051f2dc6-75b9-45c9-9204-2b8806416def\" (UID: \"051f2dc6-75b9-45c9-9204-2b8806416def\") "
Jul 25 01:35:00.835727 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.835695    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "051f2dc6-75b9-45c9-9204-2b8806416def" (UID: "051f2dc6-75b9-45c9-9204-2b8806416def"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:35:00.837340 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.837313    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f" (OuterVolumeSpecName: "kube-api-access-ppg4f") pod "051f2dc6-75b9-45c9-9204-2b8806416def" (UID: "051f2dc6-75b9-45c9-9204-2b8806416def"). InnerVolumeSpecName "kube-api-access-ppg4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:35:00.837340 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.837325    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "051f2dc6-75b9-45c9-9204-2b8806416def" (UID: "051f2dc6-75b9-45c9-9204-2b8806416def"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:35:00.937045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.936953    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppg4f\" (UniqueName: \"kubernetes.io/projected/051f2dc6-75b9-45c9-9204-2b8806416def-kube-api-access-ppg4f\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:35:00.937045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.936993    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051f2dc6-75b9-45c9-9204-2b8806416def-maas-api-tls\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:35:00.937045 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:00.937009    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051f2dc6-75b9-45c9-9204-2b8806416def-openshift-service-ca\") on node \"ip-10-0-141-233.ec2.internal\" DevicePath \"\""
Jul 25 01:35:01.586271 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.586236    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"
Jul 25 01:35:01.586271 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.586243    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s" event={"ID":"051f2dc6-75b9-45c9-9204-2b8806416def","Type":"ContainerDied","Data":"b0d1aa56e985e47a6d4f42ba4fe58bc99b4ff6ec8d74d66d3c2eff64a847f72e"}
Jul 25 01:35:01.586786 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.586312    2574 scope.go:117] "RemoveContainer" containerID="11cb8e8f7c889eedcdbd70ab3c3bb5ad18b3e0ca5215576540e26433aa91a1fd"
Jul 25 01:35:01.610071 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.610046    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:35:01.624529 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.619122    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d50ef4-67b75dbcd4-hk45s"]
Jul 25 01:35:01.934704 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.934621    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="051f2dc6-75b9-45c9-9204-2b8806416def" path="/var/lib/kubelet/pods/051f2dc6-75b9-45c9-9204-2b8806416def/volumes"
Jul 25 01:35:01.934943 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:01.934930    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7ab413-09ec-4195-abde-ecad9fac55e6" path="/var/lib/kubelet/pods/7b7ab413-09ec-4195-abde-ecad9fac55e6/volumes"
Jul 25 01:35:04.733734 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:04.733690    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:05.132943 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:05.132838    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:10.868111 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:10.868052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:10.923244 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:10.923208    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:11.161279 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:11.161176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:11.401803 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:11.401769    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:11.881291 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:11.881262    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56bf5dc956-vp2kb_bf2e7487-a852-4ee8-9c7c-f539023be05f/authorino/0.log"
Jul 25 01:35:17.855655 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:17.855618    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tsbnn_d4349849-c613-4672-8d93-e32b8b610203/manager/0.log"
Jul 25 01:35:17.984676 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:17.984634    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l4l7j_5744f514-08af-403d-b8bb-f4d009d5971b/manager/0.log"
Jul 25 01:35:18.109249 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:18.109159    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cfc5bd855-hmd7l_5eb21b43-334a-4280-ac46-6d098b104cbf/manager/0.log"
Jul 25 01:35:18.234427 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:18.234393    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hxqch_3b8ecbeb-442e-4be4-af00-9137c82048ca/server/0.log"
Jul 25 01:35:18.775920 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:18.775890    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-h8lxc_341d8c67-6c80-4531-8786-5c321c007806/copy-manifests/0.log"
Jul 25 01:35:18.794437 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:18.794404    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-h8lxc_341d8c67-6c80-4531-8786-5c321c007806/manager/0.log"
Jul 25 01:35:19.302483 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:19.302451    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29749020-mjlkz_4ed089de-2db1-4a45-ba43-809e0f31037e/cleanup/0.log"
Jul 25 01:35:19.436972 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:19.436942    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29749035-vf6dq_415c9bbc-b711-4ec3-a883-7e303bc49803/cleanup/0.log"
Jul 25 01:35:19.559612 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:19.559474    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29749050-2bggj_b475fcda-6284-4874-a184-d4272982d24a/cleanup/0.log"
Jul 25 01:35:19.690187 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:19.690150    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8x5bp_2b34ca81-b42e-4843-8c6a-a760d629edaa/postgres/0.log"
Jul 25 01:35:20.551714 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.551683    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/util/0.log"
Jul 25 01:35:20.558426 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.558397    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/pull/0.log"
Jul 25 01:35:20.564983 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.564963    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/extract/0.log"
Jul 25 01:35:20.687377 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.687348    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/util/0.log"
Jul 25 01:35:20.693396 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.693371    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/pull/0.log"
Jul 25 01:35:20.699667 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.699642    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/extract/0.log"
Jul 25 01:35:20.817066 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.816979    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/pull/0.log"
Jul 25 01:35:20.823304 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.823283    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/extract/0.log"
Jul 25 01:35:20.829897 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.829878    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/util/0.log"
Jul 25 01:35:20.951110 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.951082    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/util/0.log"
Jul 25 01:35:20.957726 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.957701    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/pull/0.log"
Jul 25 01:35:20.964279 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:20.964233    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/extract/0.log"
Jul 25 01:35:21.095854 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.095759    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56bf5dc956-vp2kb_bf2e7487-a852-4ee8-9c7c-f539023be05f/authorino/0.log"
Jul 25 01:35:21.220648 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.220615    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z6xx8_029c4571-66d7-4d3e-8f56-99b61f141dd3/manager/0.log"
Jul 25 01:35:21.348283 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.348206    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jlvbp_244aab7e-0470-452e-842b-0c972b85d65b/manager/0.log"
Jul 25 01:35:21.593687 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.593657    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w7cd8_a91d193f-4283-44b9-97df-2d50bf9e28d8/registry-server/0.log"
Jul 25 01:35:21.726366 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.726329    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl_61feac24-fe2a-44e7-a4fd-d6a21c347d9d/manager/0.log"
Jul 25 01:35:21.980194 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:21.980112    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rvwgv_43a44b7a-a867-4fc5-ad61-e3f536fc5570/manager/0.log"
Jul 25 01:35:22.387657 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:22.387564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5_d36a9860-5bec-4dc4-8da0-a8bff55d3ba7/istio-proxy/0.log"
Jul 25 01:35:22.632369 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:22.632336    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68546f88c4-ddtrh_84617057-0087-4e44-8a39-9d67dca933a7/kube-auth-proxy/0.log"
Jul 25 01:35:23.140838 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:23.140802    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:35:23.262126 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:23.262087    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77c7dfb8cc-56vkm_1ec34bf5-1799-4d44-9e40-841748a0401f/router/0.log"
Jul 25 01:35:23.644426 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:23.644395    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pngqz_3f3a0dfc-50bf-4ef2-a11e-6e56d20b12a5/main/0.log"
Jul 25 01:35:29.912739 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.912704    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt87/must-gather-cp7ck"]
Jul 25 01:35:29.913163 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913150    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70712e34-2a6a-43b5-b020-36cffa7b33b2" containerName="maas-api"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913165    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="70712e34-2a6a-43b5-b020-36cffa7b33b2" containerName="maas-api"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913181    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913186    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913193    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c48a75-b491-436e-be3b-16963c55c262" containerName="maas-api"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913200    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c48a75-b491-436e-be3b-16963c55c262" containerName="maas-api"
Jul 25 01:35:29.913211 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913210    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0898b8de-0624-434f-ba66-ac2c5bc2d905" containerName="revoke-keys"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913219    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0898b8de-0624-434f-ba66-ac2c5bc2d905" containerName="revoke-keys"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913226    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913231    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913243    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="051f2dc6-75b9-45c9-9204-2b8806416def" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913247    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="051f2dc6-75b9-45c9-9204-2b8806416def" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913253    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f4858e-9fce-4bb1-903c-412d8dbff2db" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913258    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f4858e-9fce-4bb1-903c-412d8dbff2db" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913268    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913273    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913340    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="051f2dc6-75b9-45c9-9204-2b8806416def" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913347    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913354    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c48a75-b491-436e-be3b-16963c55c262" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913360    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913368    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0898b8de-0624-434f-ba66-ac2c5bc2d905" containerName="revoke-keys"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913374    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913380    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="70712e34-2a6a-43b5-b020-36cffa7b33b2" containerName="maas-api"
Jul 25 01:35:29.913399 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913385    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f4858e-9fce-4bb1-903c-412d8dbff2db" containerName="maas-api"
Jul 25 01:35:29.913962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913454    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913459    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53be13da-b538-46c7-adff-ed07c2fad492" containerName="istio-proxy"
Jul 25 01:35:29.913962 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.913540    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d963b32-327b-41a7-8ea1-de7c990dd832" containerName="istio-proxy"
Jul 25 01:35:29.915313 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.915287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:29.918699 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.918680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:29.921964 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.921943    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt87\"/\"kube-root-ca.crt\""
Jul 25 01:35:29.922079 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.922020    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt87\"/\"openshift-service-ca.crt\""
Jul 25 01:35:29.922734 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.922687    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bt87\"/\"default-dockercfg-jr6f2\""
Jul 25 01:35:29.925672 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:29.925643    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/must-gather-cp7ck"]
Jul 25 01:35:30.022121 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.022085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jsgl\" (UniqueName: \"kubernetes.io/projected/b8959661-1a3d-47ee-86f8-35879df223cf-kube-api-access-5jsgl\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.022299 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.022210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b8959661-1a3d-47ee-86f8-35879df223cf-must-gather-output\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.123146 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.123103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jsgl\" (UniqueName: \"kubernetes.io/projected/b8959661-1a3d-47ee-86f8-35879df223cf-kube-api-access-5jsgl\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.123319 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.123197    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b8959661-1a3d-47ee-86f8-35879df223cf-must-gather-output\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.123544 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.123526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b8959661-1a3d-47ee-86f8-35879df223cf-must-gather-output\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.130841 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.130822    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jsgl\" (UniqueName: \"kubernetes.io/projected/b8959661-1a3d-47ee-86f8-35879df223cf-kube-api-access-5jsgl\") pod \"must-gather-cp7ck\" (UID: \"b8959661-1a3d-47ee-86f8-35879df223cf\") " pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.230299 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.230265    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/must-gather-cp7ck"
Jul 25 01:35:30.367154 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.366163    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/must-gather-cp7ck"]
Jul 25 01:35:30.375571 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.375552    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:35:30.775736 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:30.775692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/must-gather-cp7ck" event={"ID":"b8959661-1a3d-47ee-86f8-35879df223cf","Type":"ContainerStarted","Data":"43914230092d85be587ab273aee81de5d0bf58eef3bd4a1616a990d31275fdfe"}
Jul 25 01:35:31.204322 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:31.204286    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:31.789543 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:31.789485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/must-gather-cp7ck" event={"ID":"b8959661-1a3d-47ee-86f8-35879df223cf","Type":"ContainerStarted","Data":"5f5b6129d571f634191b2aa874fbc68d5f984930c68ef336592e19ad2b58d526"}
Jul 25 01:35:31.789748 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:31.789562    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/must-gather-cp7ck" event={"ID":"b8959661-1a3d-47ee-86f8-35879df223cf","Type":"ContainerStarted","Data":"534e47998bf0fe3fff29b9a020f9dda7360e9ad49dbc7c36de7c62fe55c27e42"}
Jul 25 01:35:31.805903 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:31.805837    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt87/must-gather-cp7ck" podStartSLOduration=2.020318632 podStartE2EDuration="2.80581717s" totalImagesPullingTime="785.498538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:35:31.803684729 +0000 UTC m=+3586.515103128" watchObservedRunningTime="2026-07-25 01:35:31.80581717 +0000 UTC m=+3586.517235569"
Jul 25 01:35:32.763110 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:32.763079    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wf7xm_16938946-1f5a-4fa0-af4c-c997a1b62d53/global-pull-secret-syncer/0.log"
Jul 25 01:35:32.822476 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:32.822445    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log"
Jul 25 01:35:32.974966 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:32.974933    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-233.ec2.internal_9b454936a2c3aaec87ddb9b7f44eca43/haproxy/0.log"
Jul 25 01:35:36.734432 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.734395    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/extract/0.log"
Jul 25 01:35:36.759993 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.759948    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/util/0.log"
Jul 25 01:35:36.782257 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.782214    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n8slm_da62d501-2e21-41c7-9d61-82a088ac7f46/pull/0.log"
Jul 25 01:35:36.837930 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.837897    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/extract/0.log"
Jul 25 01:35:36.885343 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.885314    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/util/0.log"
Jul 25 01:35:36.910001 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.909564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mfqmd_56fa2ac6-fdf6-4c8d-8507-9011420a4c73/pull/0.log"
Jul 25 01:35:36.934840 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.934810    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/extract/0.log"
Jul 25 01:35:36.955815 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.955789    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/util/0.log"
Jul 25 01:35:36.976612 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:36.976579    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bnt5z_ae843d58-08aa-4c34-9720-ddea4050b699/pull/0.log"
Jul 25 01:35:37.000675 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.000633    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/extract/0.log"
Jul 25 01:35:37.021229 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.021204    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/util/0.log"
Jul 25 01:35:37.049415 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.049372    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mk5m6_e4375d1c-f127-45b7-961c-96c979d96cb5/pull/0.log"
Jul 25 01:35:37.634535 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.632451    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56bf5dc956-vp2kb_bf2e7487-a852-4ee8-9c7c-f539023be05f/authorino/0.log"
Jul 25 01:35:37.671324 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.671286    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z6xx8_029c4571-66d7-4d3e-8f56-99b61f141dd3/manager/0.log"
Jul 25 01:35:37.698369 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.698337    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jlvbp_244aab7e-0470-452e-842b-0c972b85d65b/manager/0.log"
Jul 25 01:35:37.787716 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:37.787680    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w7cd8_a91d193f-4283-44b9-97df-2d50bf9e28d8/registry-server/0.log"
Jul 25 01:35:38.162290 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:38.162182    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qz8rl_61feac24-fe2a-44e7-a4fd-d6a21c347d9d/manager/0.log"
Jul 25 01:35:38.305032 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:38.304941    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rvwgv_43a44b7a-a867-4fc5-ad61-e3f536fc5570/manager/0.log"
Jul 25 01:35:39.873840 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.873810    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/alertmanager/0.log"
Jul 25 01:35:39.894189 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.894159    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/config-reloader/0.log"
Jul 25 01:35:39.915172 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.915141    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/kube-rbac-proxy-web/0.log"
Jul 25 01:35:39.938711 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.938658    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/kube-rbac-proxy/0.log"
Jul 25 01:35:39.958915 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.958884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/kube-rbac-proxy-metric/0.log"
Jul 25 01:35:39.987712 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:39.987669    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/prom-label-proxy/0.log"
Jul 25 01:35:40.010341 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.010306    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d4464bbd-7553-4f3c-8eab-f2b858f46b28/init-config-reloader/0.log"
Jul 25 01:35:40.061723 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.061687    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-jkkmr_1947fb06-ef29-4fa6-8630-32bafb55d402/cluster-monitoring-operator/0.log"
Jul 25 01:35:40.110372 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.110335    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v95ws_41811469-4ed5-4543-ba64-a907100bc569/kube-state-metrics/0.log"
Jul 25 01:35:40.133569 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.133466    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v95ws_41811469-4ed5-4543-ba64-a907100bc569/kube-rbac-proxy-main/0.log"
Jul 25 01:35:40.157054 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.157028    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v95ws_41811469-4ed5-4543-ba64-a907100bc569/kube-rbac-proxy-self/0.log"
Jul 25 01:35:40.232658 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.232624    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-crwnp_cb80a9ca-8027-48d1-a26f-af96708b6e79/monitoring-plugin/0.log"
Jul 25 01:35:40.359884 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.359830    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnwsz_76165ad8-fb19-412c-96f0-bb972c9d72b6/node-exporter/0.log"
Jul 25 01:35:40.398423 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.398399    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnwsz_76165ad8-fb19-412c-96f0-bb972c9d72b6/kube-rbac-proxy/0.log"
Jul 25 01:35:40.427482 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.427455    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnwsz_76165ad8-fb19-412c-96f0-bb972c9d72b6/init-textfile/0.log"
Jul 25 01:35:40.953023 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.952991    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-t6jjq_067d5345-5a0b-4d32-9127-515d3ba29144/prometheus-operator-admission-webhook/0.log"
Jul 25 01:35:40.985993 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:40.985960    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-654b776f58-5thfh_3e2896ae-8597-4f8c-a38d-02372ea1f6e4/telemeter-client/0.log"
Jul 25 01:35:41.014220 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.014165    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-654b776f58-5thfh_3e2896ae-8597-4f8c-a38d-02372ea1f6e4/reload/0.log"
Jul 25 01:35:41.042042 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.042011    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-654b776f58-5thfh_3e2896ae-8597-4f8c-a38d-02372ea1f6e4/kube-rbac-proxy/0.log"
Jul 25 01:35:41.085946 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.085918    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/thanos-query/0.log"
Jul 25 01:35:41.121747 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.121720    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/kube-rbac-proxy-web/0.log"
Jul 25 01:35:41.142483 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.142452    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/kube-rbac-proxy/0.log"
Jul 25 01:35:41.179773 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.168125    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/prom-label-proxy/0.log"
Jul 25 01:35:41.196185 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.195336    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/kube-rbac-proxy-rules/0.log"
Jul 25 01:35:41.219331 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:41.219303    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69f88cbbd9-p2fgs_5e50a877-856a-4ed5-b4e3-cfea0899f9f6/kube-rbac-proxy-metrics/0.log"
Jul 25 01:35:42.547271 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:42.547233    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-xv9g9_0fb6c2c8-80c7-467a-86b5-062613ab1f15/networking-console-plugin/0.log"
Jul 25 01:35:43.155983 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.155946    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:35:43.161212 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.161149    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/2.log"
Jul 25 01:35:43.312600 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.312566    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"]
Jul 25 01:35:43.321469 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.321434    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.325371 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.325319    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"]
Jul 25 01:35:43.381539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.381390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-podres\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.381539 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.381433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-lib-modules\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.381799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.381557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjcds\" (UniqueName: \"kubernetes.io/projected/74834647-2be3-4f68-9a45-3fe8813fc364-kube-api-access-zjcds\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.381799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.381599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-proc\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.381799 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.381706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-sys\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483222 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483173    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-sys\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-podres\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-lib-modules\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-sys\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjcds\" (UniqueName: \"kubernetes.io/projected/74834647-2be3-4f68-9a45-3fe8813fc364-kube-api-access-zjcds\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483436 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-proc\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483730 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483470    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-lib-modules\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483730 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-proc\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.483848 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.483824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74834647-2be3-4f68-9a45-3fe8813fc364-podres\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.493935 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.493906    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjcds\" (UniqueName: \"kubernetes.io/projected/74834647-2be3-4f68-9a45-3fe8813fc364-kube-api-access-zjcds\") pod \"perf-node-gather-daemonset-n57wp\" (UID: \"74834647-2be3-4f68-9a45-3fe8813fc364\") " pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.637102 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.637061    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:43.684776 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.684729    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7997d47658-s2gs2_099ece9c-3e2d-403f-a44a-4e906b1800fc/console/0.log"
Jul 25 01:35:43.763545 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.763435    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-rhg4x_64b7e67f-2fb5-43d9-af0a-23df633b13a3/download-server/0.log"
Jul 25 01:35:43.859533 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.858887    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"]
Jul 25 01:35:43.922965 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:43.922919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp" event={"ID":"74834647-2be3-4f68-9a45-3fe8813fc364","Type":"ContainerStarted","Data":"22924d480416e5bfed2fd226cb693695db0946119ad724ce2f17fa7f3bbb8ac9"}
Jul 25 01:35:44.935139 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:44.935088    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp" event={"ID":"74834647-2be3-4f68-9a45-3fe8813fc364","Type":"ContainerStarted","Data":"626fe1f62b6d36b21718ee9fd01dca649d15e1c2d6f47eac3d0095dc364c7ec1"}
Jul 25 01:35:44.935622 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:44.935163    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:44.953938 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:44.953872    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp" podStartSLOduration=1.95384939 podStartE2EDuration="1.95384939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:35:44.949301847 +0000 UTC m=+3599.660720241" watchObservedRunningTime="2026-07-25 01:35:44.95384939 +0000 UTC m=+3599.665267792"
Jul 25 01:35:45.083129 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:45.083085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-69jv9_3b5b0329-a6a0-4bae-8f69-7cdf270262d2/konnectivity-agent/0.log" limit=1024
Jul 25 01:35:45.534318 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:45.534285    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvwv5_8efb9a7a-78b0-4351-8516-1af11cbe34a5/dns/0.log"
Jul 25 01:35:45.553760 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:45.553728    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvwv5_8efb9a7a-78b0-4351-8516-1af11cbe34a5/kube-rbac-proxy/0.log"
Jul 25 01:35:45.602055 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:45.602022    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k5qjn_41a7e68e-c7eb-4150-9b35-424eaf1cb3f7/dns-node-resolver/0.log"
Jul 25 01:35:46.242542 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:46.242484    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:35:46.253754 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:46.253722    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5qb9_59310f30-69f1-4d8b-8939-2320ae3ca7b4/node-ca/0.log"
Jul 25 01:35:46.262249 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:46.262213    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zb5ns_35fece75-0dd8-41f9-a7bd-0656c707d5b5/console-operator/1.log"
Jul 25 01:35:47.181229 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:47.181190    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5_d36a9860-5bec-4dc4-8da0-a8bff55d3ba7/istio-proxy/0.log"
Jul 25 01:35:47.352209 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:47.352166    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68546f88c4-ddtrh_84617057-0087-4e44-8a39-9d67dca933a7/kube-auth-proxy/0.log"
Jul 25 01:35:47.499464 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:47.499423    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:35:47.523651 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:47.523608    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77c7dfb8cc-56vkm_1ec34bf5-1799-4d44-9e40-841748a0401f/router/0.log"
Jul 25 01:35:48.381413 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:48.381383    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzzs5_128b0291-5428-46d8-b2dd-fa8aeb05fe30/serve-healthcheck-canary/0.log"
Jul 25 01:35:48.840735 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:48.840693    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-n8sw9_00d17a42-c88c-4260-87e6-bd0d9a8618e5/insights-operator/0.log"
Jul 25 01:35:48.842036 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:48.842011    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-n8sw9_00d17a42-c88c-4260-87e6-bd0d9a8618e5/insights-operator/1.log"
Jul 25 01:35:48.984290 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:48.984260    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvxkg_f7e3e027-f042-4489-8d36-711e9fe930eb/kube-rbac-proxy/0.log"
Jul 25 01:35:49.002339 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:49.002308    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvxkg_f7e3e027-f042-4489-8d36-711e9fe930eb/exporter/0.log"
Jul 25 01:35:49.023411 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:49.023375    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvxkg_f7e3e027-f042-4489-8d36-711e9fe930eb/extractor/0.log"
Jul 25 01:35:50.960579 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:50.960540    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bt87/perf-node-gather-daemonset-n57wp"
Jul 25 01:35:51.096077 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:51.096041    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tsbnn_d4349849-c613-4672-8d93-e32b8b610203/manager/0.log"
Jul 25 01:35:51.511557 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:51.511529    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l4l7j_5744f514-08af-403d-b8bb-f4d009d5971b/manager/0.log"
Jul 25 01:35:51.743734 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:51.743625    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cfc5bd855-hmd7l_5eb21b43-334a-4280-ac46-6d098b104cbf/manager/0.log"
Jul 25 01:35:51.767655 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:51.767577    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hxqch_3b8ecbeb-442e-4be4-af00-9137c82048ca/server/0.log"
Jul 25 01:35:52.100467 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:52.100396    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-h8lxc_341d8c67-6c80-4531-8786-5c321c007806/manager/0.log"
Jul 25 01:35:52.116541 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:52.116497    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b8579fc9-h8lxc_341d8c67-6c80-4531-8786-5c321c007806/copy-manifests/0.log"
Jul 25 01:35:54.394858 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:54.394812    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5_d36a9860-5bec-4dc4-8da0-a8bff55d3ba7/istio-proxy/0.log"
Jul 25 01:35:54.538258 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:54.538219    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68546f88c4-ddtrh_84617057-0087-4e44-8a39-9d67dca933a7/kube-auth-proxy/0.log"
Jul 25 01:35:54.648134 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:54.648093    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:35:54.665869 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:54.665840    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77c7dfb8cc-56vkm_1ec34bf5-1799-4d44-9e40-841748a0401f/router/0.log"
Jul 25 01:35:56.068827 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:56.068740    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hbmn5_d36a9860-5bec-4dc4-8da0-a8bff55d3ba7/istio-proxy/0.log"
Jul 25 01:35:56.225674 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:56.225640    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68546f88c4-ddtrh_84617057-0087-4e44-8a39-9d67dca933a7/kube-auth-proxy/0.log"
Jul 25 01:35:56.340919 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:56.340824    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wmwsl_97665a0e-e7d9-4062-9152-6d17133eb669/payload-processing/0.log"
Jul 25 01:35:56.361847 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:56.361816    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77c7dfb8cc-56vkm_1ec34bf5-1799-4d44-9e40-841748a0401f/router/0.log"
Jul 25 01:35:58.949742 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:58.949708    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6_b0b9f0f3-700d-4804-9885-87868abc05df/kube-storage-version-migrator-operator/1.log"
Jul 25 01:35:58.951122 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:35:58.951068    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-bvjf6_b0b9f0f3-700d-4804-9885-87868abc05df/kube-storage-version-migrator-operator/0.log"
Jul 25 01:36:00.056536 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.056491    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/kube-multus-additional-cni-plugins/0.log"
Jul 25 01:36:00.075827 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.075797    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/egress-router-binary-copy/0.log"
Jul 25 01:36:00.095185 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.095153    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/cni-plugins/0.log"
Jul 25 01:36:00.114326 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.114300    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/bond-cni-plugin/0.log"
Jul 25 01:36:00.134580 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.134541    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/routeoverride-cni/0.log"
Jul 25 01:36:00.154095 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.154065    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/whereabouts-cni-bincopy/0.log"
Jul 25 01:36:00.173526 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.173489    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5kdf_b5e52a4f-42bd-4e53-b34c-a17c2b068f12/whereabouts-cni/0.log"
Jul 25 01:36:00.370195 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.370107    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g9xqk_90764f53-efa3-453c-9e7d-0a38b182318b/kube-multus/0.log"
Jul 25 01:36:00.507472 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.507442    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wdr58_c67c6b28-9884-4498-bd1f-0b5f7d52231f/network-metrics-daemon/0.log"
Jul 25 01:36:00.520949 ip-10-0-141-233 kubenswrapper[2574]: I0725 01:36:00.520926    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wdr58_c67c6b28-9884-4498-bd1f-0b5f7d52231f/kube-rbac-proxy/0.log"