Jul 28 23:19:57.158210 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 23:19:57.158220 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 23:19:57.158228 ip-10-0-135-12 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 23:19:57.158493 ip-10-0-135-12 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 23:20:07.398010 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 23:20:07.398025 ip-10-0-135-12 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b200e0f0d7024c00bc3e78d1c1901c6c --
Jul 28 23:21:39.344649 ip-10-0-135-12 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 23:21:39.793443 ip-10-0-135-12 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:21:39.793443 ip-10-0-135-12 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 23:21:39.793443 ip-10-0-135-12 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:21:39.793443 ip-10-0-135-12 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 23:21:39.793443 ip-10-0-135-12 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:21:39.795466 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.795367    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803706    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803730    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803735    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803738    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803741    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803746    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803750    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.803740 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803753    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803756    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803759    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803762    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803766    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803771    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803774    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803777    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803779    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803782    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803785    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803788    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803790    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803793    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803796    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803799    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803801    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803804    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803813    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.804031 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803817    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803819    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803822    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803824    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803827    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803829    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803832    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803834    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803836    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803839    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803842    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803845    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803847    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803849    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803852    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803856    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803859    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803862    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803864    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803867    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.804498 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803869    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803873    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803876    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803878    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803881    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803883    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803886    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803888    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803891    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803893    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803896    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803900    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803904    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803907    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803910    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803913    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803915    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803918    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803921    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803923    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.805005 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803926    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803928    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803931    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803934    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803936    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803939    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803942    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803945    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803947    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803950    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803952    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803955    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803957    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803960    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803962    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803965    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803967    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803973    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803976    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803978    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.805527 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803981    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803983    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803986    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803988    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803991    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803994    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.803998    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804002    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804005    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804008    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804010    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804013    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804016    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804739    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804746    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804749    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804752    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804755    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804758    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.806011 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804761    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804764    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804766    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804770    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804772    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804775    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804778    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804781    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804783    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804785    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804788    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804791    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804794    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804796    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804798    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804801    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804805    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804808    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804811    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.806484 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804814    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804817    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804819    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804822    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804825    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804828    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804831    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804835    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804838    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804842    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804845    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804848    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804851    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804854    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804857    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804860    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804863    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804866    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804869    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.806947 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804871    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804874    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804876    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804880    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804882    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804885    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804888    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804891    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804894    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804896    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804898    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804901    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804903    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804906    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804908    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804911    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804914    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804918    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804921    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.807428 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804924    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804927    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804929    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804932    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804934    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804938    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804941    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804944    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804946    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804949    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804952    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804954    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804958    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804961    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804963    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804966    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804968    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804970    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804973    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804976    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.807892 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804979    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804981    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804984    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804986    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804989    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804992    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804994    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.804997    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805000    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805002    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805005    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805008    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805010    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805013    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805016    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805019    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805441    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805447    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805450    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805453    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.808379 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805456    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805459    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805461    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805464    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805467    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805469    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805472    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805475    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805477    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805480    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805483    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805485    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805489    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805491    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805494    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805497    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805499    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805502    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805504    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.808877 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805507    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805509    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805512    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805515    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805518    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805520    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805523    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805528    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805532    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805535    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805539    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805542    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805545    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805547    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805550    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805552    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805555    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805557    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805560    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805562    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.809328 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805565    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805567    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805570    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805572    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805575    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805577    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805580    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805583    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805585    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805588    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805590    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805593    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805595    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805598    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805600    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805603    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805606    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805608    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805611    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805614    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.809831 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805617    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805619    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805621    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805624    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805626    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805630    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805633    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805635    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805638    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805640    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805643    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805645    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805649    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805652    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805655    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805658    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805661    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805664    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805667    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.810338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805670    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805673    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805676    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805679    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805681    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805684    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805687    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805689    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805692    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805695    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805697    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805700    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805703    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805705    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805708    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805710    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.805712    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805786    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805798    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805806    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805812    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 23:21:39.810807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805818    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805822    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805827    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805831    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805834    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805837    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805841    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805844    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805847    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805850    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805853    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805855    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805859    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805861    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805866    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805868    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805871    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805874    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805878    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805884    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805887    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805890    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805894    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805897    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 23:21:39.811310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805900    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805903    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805907    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805909    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805914    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805917    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805920    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805924    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805927    2564 flags.go:64] FLAG: --enable-server="true"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805930    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805934    2564 flags.go:64] FLAG: --event-burst="100"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805938    2564 flags.go:64] FLAG: --event-qps="50"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805941    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805944    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805947    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805950    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805953    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805956    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805959    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805962    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805965    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805968    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805971    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805974    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805977    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 23:21:39.811949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805980    2564 flags.go:64] FLAG: --feature-gates=""
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805984    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805988    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805991    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805994    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.805997    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806001    2564 flags.go:64] FLAG: --help="false"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806003    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-135-12.ec2.internal"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806006    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806010    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806013    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806017    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806020    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806023    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806026    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806029    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806032    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806035    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806038    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806041    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806044    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806046    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806049    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806052    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 23:21:39.812585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806055    2564 flags.go:64] FLAG: --lock-file=""
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806058    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806061    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806064    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806069    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806073    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806076    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806079    2564 flags.go:64] FLAG: --logging-format="text"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806081    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806085    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806089    2564 flags.go:64] FLAG: --manifest-url=""
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806092    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806097    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806100    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806104    2564 flags.go:64] FLAG: --max-pods="110"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806107    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806110    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806113    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806116    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806119    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806121    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806124    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806132    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806135    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806138    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 23:21:39.813163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806141    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806144    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806150    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806153    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806157    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806160    2564 flags.go:64] FLAG: --port="10250"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806162    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806166    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04982287c92adeb0b"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806169    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806171    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806174    2564 flags.go:64] FLAG: --register-node="true"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806177    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806180    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806184    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806187    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806190    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806196    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806200    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806203    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806206    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806209    2564 flags.go:64] FLAG: --runonce="false"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806212    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806215    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806218    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806221    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 23:21:39.813848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806224    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806227    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806230    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806233    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806236    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806239    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806242    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806245    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806248    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806251    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806253    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806259    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806262    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806264    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806268    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806271    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806274    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806277    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806279    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806282    2564 flags.go:64] FLAG: --v="2"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806287    2564 flags.go:64] FLAG: --version="false"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806291    2564 flags.go:64] FLAG: --vmodule=""
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806295    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.806300    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 23:21:39.814473 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806431    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806436    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806440    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806443    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806446    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806449    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806452    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806454    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806457    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806460    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806463    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806466    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806469    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806471    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806474    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806477    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806480    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806482    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806485    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.815097 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806488    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806491    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806494    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806497    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806499    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806502    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806504    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806507    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806510    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806512    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806514    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806517    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806521    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806525    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806528    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806530    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806533    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806535    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806538    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806540    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.815586 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806543    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806546    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806548    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806551    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806553    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806556    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806558    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806561    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806564    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806566    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806569    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806571    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806574    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806576    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806579    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806581    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806584    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806586    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806589    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806592    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.816066 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806595    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806597    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806600    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806602    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806609    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806614    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806617    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806619    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806622    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806625    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806628    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806631    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806633    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806636    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806639    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806642    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806645    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806648    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806651    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.816564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806653    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806656    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806659    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806661    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806664    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806666    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806670    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806672    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806675    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806677    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806680    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806682    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806685    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806689    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806692    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806695    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806698    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806702    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806706    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806709    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.817030 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.806712    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.807490    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.815076    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.815094    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.816012    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.816020    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816078    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816084    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816088    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816091    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816094    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816097    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816100    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816102    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.817532 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816105    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816108    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816111    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816113    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816116    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816119    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816121    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816124    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816126    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816129    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816132    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816134    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816137    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816140    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816143    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816146    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816150    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816153    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816156    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816159    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.817880 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816162    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816164    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816168    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816170    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816173    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816176    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816178    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816181    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816183    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816186    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816189    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816191    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816194    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816197    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816200    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816202    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816205    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816208    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816210    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816213    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.818386 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816215    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816218    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816221    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816224    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816227    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816230    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816232    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816235    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816238    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816241    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816243    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816246    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816249    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816251    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816254    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816257    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816259    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816262    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816264    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816267    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.818865 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816269    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816272    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816274    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816277    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816279    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816282    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816285    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816287    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816290    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816294    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816297    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816300    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816303    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816306    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816309    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816311    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816315    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816318    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816321    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.819354 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816325    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816329    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816331    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816335    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816338    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816354    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816359    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816363    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816366    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816369    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816373    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816377    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.816383    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816493    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816497    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:21:39.819847 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816500    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816503    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816506    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816509    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816512    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816515    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816518    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816521    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816524    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816526    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816529    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816531    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816534    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816538    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816542    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816545    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816548    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816551    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816553    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:21:39.820211 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816556    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816559    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816562    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816565    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816567    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816570    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816573    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816576    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816579    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816582    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816584    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816586    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816589    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816592    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816594    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816597    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816599    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816602    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816605    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816607    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:21:39.820688 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816610    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816612    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816615    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816617    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816620    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816622    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816625    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816628    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816631    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816635    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816637    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816640    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816643    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816645    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816648    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816651    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816653    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816656    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816658    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:21:39.821161 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816661    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816664    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816666    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816668    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816671    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816673    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816676    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816678    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816681    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816683    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816685    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816688    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816690    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816693    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816695    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816698    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816700    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816703    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816705    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:21:39.821654 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816708    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816710    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816713    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816717    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816721    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816724    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816727    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816730    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816733    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816736    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816740    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816743    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816745    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816748    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816750    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816753    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816755    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816758    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816761    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:21:39.822105 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:39.816763    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.816768    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.816889    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.819313    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.821093    2564 server.go:1019] "Starting client certificate rotation"
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.821195    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 23:21:39.822623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.821235    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 23:21:39.823981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.823967    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 23:21:39.828238 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.828213    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 23:21:39.840127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.840103    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 28 23:21:39.842641 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.842622    2564 log.go:25] "Validated CRI v1 image API"
Jul 28 23:21:39.843414 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.843401    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 23:21:39.846912 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.846887    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9b1fd377-50b5-45ba-b146-81bffdb7a97e:/dev/nvme0n1p3 fc5f9af5-1197-438f-ae8e-c2f67333e5c1:/dev/nvme0n1p4]
Jul 28 23:21:39.846980 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.846913    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 23:21:39.848559 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.848540    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 23:21:39.853254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.853152    2564 manager.go:216] Machine: {Timestamp:2026-07-28 23:21:39.851842869 +0000 UTC m=+0.378892123 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101767 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b2fbd24ff4cefe670bed557c5e68f SystemUUID:ec2b2fbd-24ff-4cef-e670-bed557c5e68f BootID:b200e0f0-d702-4c00-bc3e-78d1c1901c6c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ae:a5:cd:92:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ae:a5:cd:92:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:bd:b7:9c:32:a0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 23:21:39.853254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.853249    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 23:21:39.853369 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.853331    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 23:21:39.854574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.854551    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 23:21:39.854711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.854577    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 23:21:39.855578 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.855568    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 23:21:39.855611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.855581    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 23:21:39.855611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.855588    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 23:21:39.855611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.855602    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 23:21:39.856801 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.856791    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 23:21:39.859596 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859585    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 23:21:39.859707 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859697    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 23:21:39.859749 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859728    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 23:21:39.859749 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859738    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 23:21:39.859749 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859749    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 23:21:39.859861 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859758    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 28 23:21:39.859861 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.859767    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 23:21:39.860110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.860098    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 23:21:39.860145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.860119    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 23:21:39.862456 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.862438    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 23:21:39.864879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.864865    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 23:21:39.864944 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.864895    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 23:21:39.866858 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866847    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866863    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866869    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866875    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866883    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866889    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866896    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 23:21:39.866905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866908    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 23:21:39.867162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866915    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 23:21:39.867162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866920    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 23:21:39.867162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866932    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 23:21:39.867162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.866937    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 23:21:39.868509 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.868495    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 23:21:39.868509 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.868504    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 23:21:39.869769 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.869527    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 23:21:39.869855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.869675    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 23:21:39.870031 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.869982    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 23:21:39.872244 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.872229    2564 server.go:1268] "Started kubelet"
Jul 28 23:21:39.872334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.872306    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 23:21:39.872404 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.872314    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 23:21:39.872452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.872406    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 23:21:39.873112 ip-10-0-135-12 systemd[1]: Started Kubernetes Kubelet.
Jul 28 23:21:39.873481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.873466    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 23:21:39.874633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.874616    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 23:21:39.878086 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.878066    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 23:21:39.878275 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.877152    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-12.ec2.internal.18c69785052bfcc7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-12.ec2.internal,UID:ip-10-0-135-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-12.ec2.internal,},FirstTimestamp:2026-07-28 23:21:39.872193735 +0000 UTC m=+0.399242988,LastTimestamp:2026-07-28 23:21:39.872193735 +0000 UTC m=+0.399242988,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-12.ec2.internal,}"
Jul 28 23:21:39.884413 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.884393    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 23:21:39.884513 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.884410    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 23:21:39.884580 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.884525    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:39.885141 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885117    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 23:21:39.885141 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885140    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 23:21:39.885298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885143    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 23:21:39.885298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885181    2564 factory.go:55] Registering systemd factory
Jul 28 23:21:39.885298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885198    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 28 23:21:39.885298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885254    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 23:21:39.885298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885264    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 23:21:39.885549 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885535    2564 factory.go:153] Registering CRI-O factory
Jul 28 23:21:39.885599 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885553    2564 factory.go:223] Registration of the crio container factory successfully
Jul 28 23:21:39.885651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885603    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 23:21:39.885651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885626    2564 factory.go:103] Registering Raw factory
Jul 28 23:21:39.885651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885641    2564 manager.go:1196] Started watching for new ooms in manager
Jul 28 23:21:39.886012 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.885994    2564 manager.go:349] Starting recovery of all containers
Jul 28 23:21:39.892895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.892872    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pdcq"
Jul 28 23:21:39.896188 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.896166    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 23:21:39.896275 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.896221    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 23:21:39.897612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.897598    2564 manager.go:354] Recovery completed
Jul 28 23:21:39.899072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.899049    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pdcq"
Jul 28 23:21:39.904626 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.904614    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:39.907265 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907250    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:39.907328 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907276    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:39.907328 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907286    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:39.907882 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907867    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 23:21:39.907882 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907882    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 23:21:39.907997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907893    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:39.907997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.907900    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 23:21:39.910074 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910055    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:39.910143 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910092    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:39.910143 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910107    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:39.910633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910620    2564 policy_none.go:49] "None policy: Start"
Jul 28 23:21:39.910696 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910639    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 23:21:39.910696 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.910652    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 23:21:39.912667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.912654    2564 policy_none.go:47] "Start"
Jul 28 23:21:39.958015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.957992    2564 manager.go:337] "Starting Device Plugin manager"
Jul 28 23:21:39.958134 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.958091    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 23:21:39.958134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958106    2564 server.go:87] "Starting device plugin registration server"
Jul 28 23:21:39.958393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958380    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 23:21:39.958430 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958394    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 23:21:39.958494 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958480    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 23:21:39.958597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958556    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 23:21:39.958597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.958564    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 23:21:39.959108 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.959087    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 23:21:39.959192 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:39.959144    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:39.976776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:39.976751    2564 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2594 into the system container \"/system.slice\": <nil>"
Jul 28 23:21:40.059067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.059007    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:40.059997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.059979    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:40.060105 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.060013    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:40.060105 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.060028    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:40.060105 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.060058    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.069844 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.069673    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.083854 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.069852    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-12.ec2.internal\": node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.095906 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.095887    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.107264 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.107235    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 23:21:40.108425 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.108407    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 23:21:40.108508 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.108444    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 23:21:40.108508 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.108464    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 23:21:40.108508 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.108494    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 23:21:40.110612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.110596    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:21:40.196740 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.196701    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.208873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.208846    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"]
Jul 28 23:21:40.208950 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.208940    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:40.212577 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.212561    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:40.212681 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.212591    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:40.212681 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.212601    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:40.214991 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.214978    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:40.215125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215110    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.215186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215148    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:40.215779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215763    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:40.215849 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215770    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:40.215849 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215813    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:40.215849 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215823    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:40.215934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215790    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:40.215934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.215875    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:40.218107 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.218086    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.218196 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.218117    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:21:40.218811 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.218796    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:21:40.218868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.218826    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:21:40.218868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.218842    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:21:40.236981 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.236958    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-12.ec2.internal\" not found" node="ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.240970 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.240956    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-12.ec2.internal\" not found" node="ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.287433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.287402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.287433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.287437    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.287624 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.287454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39abfa21be29f5a114b5174162890ec9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"39abfa21be29f5a114b5174162890ec9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.297010 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.296992    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.387798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.387798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.387798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39abfa21be29f5a114b5174162890ec9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"39abfa21be29f5a114b5174162890ec9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.387987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/39abfa21be29f5a114b5174162890ec9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"39abfa21be29f5a114b5174162890ec9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.387987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.387987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.387866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21ee4b15ed3fe9ce757cd4b6354bec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"5d21ee4b15ed3fe9ce757cd4b6354bec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.397796 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.397774    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.498490 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.498465    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.541970 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.541924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.544242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.544225    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:40.599101 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.599066    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.700030 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.699937    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.800161 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.800125    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.820612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.820587    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 23:21:40.820731 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.820711    2564 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.135.12:45508->54.152.146.177:6443: use of closed network connection"
Jul 28 23:21:40.820814 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.820786    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 23:21:40.820911 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.820892    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: unexpected error when reading response body. Please retry. Original error: read tcp 10.0.135.12:45508->54.152.146.177:6443: use of closed network connection" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 23:21:40.900308 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:40.900268    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:40.906984 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.906940    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 23:16:40 +0000 UTC" deadline="2028-04-20 13:57:27.778534098 +0000 UTC"
Jul 28 23:21:40.906984 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.906983    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15158h35m46.871554447s"
Jul 28 23:21:40.981614 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:40.981526    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:21:41.001281 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.001244    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:41.101805 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.101775    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:41.202158 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.202125    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:41.250129 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:41.250091    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39abfa21be29f5a114b5174162890ec9.slice/crio-15e365197fa83773faf37d8bbc456b13e29a63fd7055da65536ce5d458e6f1d1 WatchSource:0}: Error finding container 15e365197fa83773faf37d8bbc456b13e29a63fd7055da65536ce5d458e6f1d1: Status 404 returned error can't find the container with id 15e365197fa83773faf37d8bbc456b13e29a63fd7055da65536ce5d458e6f1d1
Jul 28 23:21:41.250466 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:41.250438    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d21ee4b15ed3fe9ce757cd4b6354bec.slice/crio-6058224295b182558040d9b88e1a6fe9b3577cbcb9cf9be45b54faf2baa5741c WatchSource:0}: Error finding container 6058224295b182558040d9b88e1a6fe9b3577cbcb9cf9be45b54faf2baa5741c: Status 404 returned error can't find the container with id 6058224295b182558040d9b88e1a6fe9b3577cbcb9cf9be45b54faf2baa5741c
Jul 28 23:21:41.254879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.254862    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:21:41.302785 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.302740    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:41.403413 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.403377    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found"
Jul 28 23:21:41.436417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.436394    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:21:41.485623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.485595    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:41.498198 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.498143    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 23:21:41.500072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.500058    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal"
Jul 28 23:21:41.511193 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.511173    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 23:21:41.741070 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.741041    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:21:41.861503 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.861404    2564 apiserver.go:52] "Watching apiserver"
Jul 28 23:21:41.872714 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.872676    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 23:21:41.874746 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.874683    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-f4k6f","kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal","openshift-dns/node-resolver-zwzzw","openshift-image-registry/node-ca-h2ts6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal","openshift-multus/network-metrics-daemon-bw89j","openshift-network-diagnostics/network-check-target-djhbc","openshift-network-operator/iptables-alerter-mzxj8","kube-system/konnectivity-agent-4bvx6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg","openshift-cluster-node-tuning-operator/tuned-x7rxd","openshift-multus/multus-additional-cni-plugins-8cfrg","openshift-multus/multus-gwzhj","openshift-ovn-kubernetes/ovnkube-node-44zbm"]
Jul 28 23:21:41.877423 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.877401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:41.877543 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.877486    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:41.879835 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.879732    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.882374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.882191    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.882374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.882280    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.882570 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.882399    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvz2b\""
Jul 28 23:21:41.882770 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.882755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:41.885183 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.885135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.885290 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.885216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:41.885517 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.885492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.885608 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.885583    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gw6n9\""
Jul 28 23:21:41.885715 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.885700    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 23:21:41.887780 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.887758    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.887974 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.887953    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.888385 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.888359    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v2hlg\""
Jul 28 23:21:41.888522 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.888370    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.888522 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.888462    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 23:21:41.890237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.890217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:41.890338 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.890295    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:41.890512 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.890464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 23:21:41.890694 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.890677    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 23:21:41.890952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.890930    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.892012 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.891991    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 23:21:41.892275 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.892257    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.892531 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.892505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gq66v\""
Jul 28 23:21:41.892700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.892679    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 23:21:41.893876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.892928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:41.895824 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.895804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:41.897028 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.896851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 23:21:41.897028 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.896896    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 23:21:41.897191 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.897127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bxrhq\""
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.897924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-systemd-units\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.897956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-netns\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.897982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.897985    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-config\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-host\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-slash\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-systemd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-log-socket\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-bin\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-netd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-kubernetes\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-run\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.898255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7bac126-cd63-4eb0-9e70-77bd8be227f8-host\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-var-lib-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-node-log\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-script-lib\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-tuned\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cf8882-071f-4e89-9728-13511310389b-tmp-dir\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-etc-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-conf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-lib-modules\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-tmp\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898510    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qlptt\""
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78cf8882-071f-4e89-9728-13511310389b-hosts-file\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898558    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7bac126-cd63-4eb0-9e70-77bd8be227f8-serviceca\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-kubelet\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.898774    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899129    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 23:21:41.899158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-ovn\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899182    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-modprobe-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-systemd\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-env-overrides\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/083244cb-7b2a-4780-8636-faf88f1304da-ovn-node-metrics-cert\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svbxk\" (UniqueName: \"kubernetes.io/projected/083244cb-7b2a-4780-8636-faf88f1304da-kube-api-access-svbxk\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysconfig\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899440    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-sys\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wjh\" (UniqueName: \"kubernetes.io/projected/c7bac126-cd63-4eb0-9e70-77bd8be227f8-kube-api-access-j4wjh\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-var-lib-kubelet\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wsnf\" (UniqueName: \"kubernetes.io/projected/4b0906a2-dc46-42c2-845a-a87c3d117a1b-kube-api-access-2wsnf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:41.900037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.899513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xbvd\" (UniqueName: \"kubernetes.io/projected/78cf8882-071f-4e89-9728-13511310389b-kube-api-access-2xbvd\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:41.900634 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.900062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:41.902450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.902427    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:41.903049 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.903021    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.903166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.903058    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 23:21:41.903166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.903102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tnkm8\""
Jul 28 23:21:41.903166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.903139    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 23:21:41.904030 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.904010    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.905076 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.904899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 23:21:41.905175 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.905077    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 23:21:41.905504 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.905483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 23:21:41.905575 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.905502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xgjcc\""
Jul 28 23:21:41.906138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.906119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:41.906215 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:41.906182    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:41.909613 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.909058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:41.911584 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.911561    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 23:21:41.911757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.911737    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 23:21:41.911831 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.911822    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx6z4\""
Jul 28 23:21:41.919573 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.919546    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-85bct"
Jul 28 23:21:41.928094 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.928069    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-85bct"
Jul 28 23:21:41.986465 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:41.986435    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-run\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-var-lib-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-system-cni-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-device-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-system-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-tuned\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c96c4a66-6319-41f3-b325-25bbdbff0d52-konnectivity-ca\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-socket-dir-parent\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.000186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-k8s-cni-cncf-io\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-lib-modules\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-run\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-tmp\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78cf8882-071f-4e89-9728-13511310389b-hosts-file\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7bac126-cd63-4eb0-9e70-77bd8be227f8-serviceca\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-kubelet\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-ovn\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-dbus\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-os-release\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78cf8882-071f-4e89-9728-13511310389b-hosts-file\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-var-lib-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/083244cb-7b2a-4780-8636-faf88f1304da-ovn-node-metrics-cert\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-os-release\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv5rr\" (UniqueName: \"kubernetes.io/projected/3fcbd120-7554-4f46-a29c-1a32bb6155bc-kube-api-access-bv5rr\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000573    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysconfig\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-lib-modules\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.000876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-sys\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wjh\" (UniqueName: \"kubernetes.io/projected/c7bac126-cd63-4eb0-9e70-77bd8be227f8-kube-api-access-j4wjh\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-ovn\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-kubelet-config\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000712    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-kubelet\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cnibin\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-hostroot\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysconfig\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-var-lib-kubelet\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wsnf\" (UniqueName: \"kubernetes.io/projected/4b0906a2-dc46-42c2-845a-a87c3d117a1b-kube-api-access-2wsnf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xbvd\" (UniqueName: \"kubernetes.io/projected/78cf8882-071f-4e89-9728-13511310389b-kube-api-access-2xbvd\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-netns\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gt4k\" (UniqueName: \"kubernetes.io/projected/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-kube-api-access-4gt4k\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7bac126-cd63-4eb0-9e70-77bd8be227f8-serviceca\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.000977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-sys\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.001884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-host\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-log-socket\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-bin\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001104    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-sys-fs\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-var-lib-kubelet\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cni-binary-copy\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-kubernetes\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-kubernetes\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb8dr\" (UniqueName: \"kubernetes.io/projected/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-kube-api-access-wb8dr\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001314    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7bac126-cd63-4eb0-9e70-77bd8be227f8-host\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-node-log\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-script-lib\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.002785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-bin\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cf8882-071f-4e89-9728-13511310389b-tmp-dir\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001490    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-host\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-etc-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-log-socket\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-socket-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-node-log\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-bin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7bac126-cd63-4eb0-9e70-77bd8be227f8-host\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001650    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-etc-openvswitch\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-multus-certs\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-conf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-env-overrides\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-sysctl-conf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.003664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-daemon-config\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-modprobe-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-systemd\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svbxk\" (UniqueName: \"kubernetes.io/projected/083244cb-7b2a-4780-8636-faf88f1304da-kube-api-access-svbxk\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-modprobe-d\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmpb2\" (UniqueName: \"kubernetes.io/projected/5f155a88-e612-4fbd-a32e-906d0e74ce6c-kube-api-access-cmpb2\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fcbd120-7554-4f46-a29c-1a32bb6155bc-iptables-alerter-script\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-systemd\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cf8882-071f-4e89-9728-13511310389b-tmp-dir\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.001975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gg5\" (UniqueName: \"kubernetes.io/projected/81c049c3-c1fd-411d-8631-18d11ec594a4-kube-api-access-q9gg5\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-kubelet\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fcbd120-7554-4f46-a29c-1a32bb6155bc-host-slash\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-registration-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.004587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cnibin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-conf-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-systemd-units\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-netns\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-config\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-etc-kubernetes\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-slash\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-systemd-units\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-systemd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-env-overrides\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-netd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-script-lib\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-cni-netd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-run-systemd\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-run-netns\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.005520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/083244cb-7b2a-4780-8636-faf88f1304da-host-slash\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c96c4a66-6319-41f3-b325-25bbdbff0d52-agent-certs\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002477    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-multus\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.002732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/083244cb-7b2a-4780-8636-faf88f1304da-ovnkube-config\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.005067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/083244cb-7b2a-4780-8636-faf88f1304da-ovn-node-metrics-cert\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.005222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-etc-tuned\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.006381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.005271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0906a2-dc46-42c2-845a-a87c3d117a1b-tmp\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.009148 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.009125    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:42.009248 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.009153    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:42.009248 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.009166    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:42.009248 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.009239    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:42.509219256 +0000 UTC m=+3.036268526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:42.012343 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.012240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wjh\" (UniqueName: \"kubernetes.io/projected/c7bac126-cd63-4eb0-9e70-77bd8be227f8-kube-api-access-j4wjh\") pod \"node-ca-h2ts6\" (UID: \"c7bac126-cd63-4eb0-9e70-77bd8be227f8\") " pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.012343 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.012241    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wsnf\" (UniqueName: \"kubernetes.io/projected/4b0906a2-dc46-42c2-845a-a87c3d117a1b-kube-api-access-2wsnf\") pod \"tuned-x7rxd\" (UID: \"4b0906a2-dc46-42c2-845a-a87c3d117a1b\") " pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.014284 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.014262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xbvd\" (UniqueName: \"kubernetes.io/projected/78cf8882-071f-4e89-9728-13511310389b-kube-api-access-2xbvd\") pod \"node-resolver-zwzzw\" (UID: \"78cf8882-071f-4e89-9728-13511310389b\") " pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.014490 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.014468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svbxk\" (UniqueName: \"kubernetes.io/projected/083244cb-7b2a-4780-8636-faf88f1304da-kube-api-access-svbxk\") pod \"ovnkube-node-44zbm\" (UID: \"083244cb-7b2a-4780-8636-faf88f1304da\") " pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.102903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.102863    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-netns\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.102921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gt4k\" (UniqueName: \"kubernetes.io/projected/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-kube-api-access-4gt4k\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.102959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.102988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-sys-fs\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cni-binary-copy\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.103080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103074    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb8dr\" (UniqueName: \"kubernetes.io/projected/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-kube-api-access-wb8dr\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-socket-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-bin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-multus-certs\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-daemon-config\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmpb2\" (UniqueName: \"kubernetes.io/projected/5f155a88-e612-4fbd-a32e-906d0e74ce6c-kube-api-access-cmpb2\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fcbd120-7554-4f46-a29c-1a32bb6155bc-iptables-alerter-script\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gg5\" (UniqueName: \"kubernetes.io/projected/81c049c3-c1fd-411d-8631-18d11ec594a4-kube-api-access-q9gg5\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.103400 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-kubelet\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fcbd120-7554-4f46-a29c-1a32bb6155bc-host-slash\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-registration-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cnibin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-conf-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-etc-kubernetes\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c96c4a66-6319-41f3-b325-25bbdbff0d52-agent-certs\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-multus\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-system-cni-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-device-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103798    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-system-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c96c4a66-6319-41f3-b325-25bbdbff0d52-konnectivity-ca\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-socket-dir-parent\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-k8s-cni-cncf-io\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-dbus\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103910    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-os-release\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-os-release\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv5rr\" (UniqueName: \"kubernetes.io/projected/3fcbd120-7554-4f46-a29c-1a32bb6155bc-kube-api-access-bv5rr\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.103978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-kubelet-config\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cnibin\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-hostroot\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-hostroot\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104132    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-netns\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.104925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.104891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-sys-fs\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.105500 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.105435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cni-binary-copy\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.105918 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.105894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cni-binary-copy\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.106277 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-cnibin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-multus-certs\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-bin\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106587    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fcbd120-7554-4f46-a29c-1a32bb6155bc-host-slash\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106593    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-kubelet\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-registration-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-socket-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106788    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f155a88-e612-4fbd-a32e-906d0e74ce6c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106792    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.106829    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-os-release\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-os-release\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.106904    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:42.606885739 +0000 UTC m=+3.133934990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-socket-dir-parent\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-daemon-config\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-run-k8s-cni-cncf-io\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.107393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-kubelet-config\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.106971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-host-var-lib-cni-multus\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-cnibin\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f155a88-e612-4fbd-a32e-906d0e74ce6c-system-cni-dir\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afaeb2da-9369-4aea-b14f-baf82476d5d0-dbus\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-multus-conf-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-system-cni-dir\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-device-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c96c4a66-6319-41f3-b325-25bbdbff0d52-konnectivity-ca\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.107363    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.107461    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:42.607440454 +0000 UTC m=+3.134489699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81c049c3-c1fd-411d-8631-18d11ec594a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.107512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-etc-kubernetes\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.108485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.108209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fcbd120-7554-4f46-a29c-1a32bb6155bc-iptables-alerter-script\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.114074 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.114017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gt4k\" (UniqueName: \"kubernetes.io/projected/ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3-kube-api-access-4gt4k\") pod \"multus-gwzhj\" (UID: \"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3\") " pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.116683 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.116533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" event={"ID":"39abfa21be29f5a114b5174162890ec9","Type":"ContainerStarted","Data":"15e365197fa83773faf37d8bbc456b13e29a63fd7055da65536ce5d458e6f1d1"}
Jul 28 23:21:42.117673 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.117613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmpb2\" (UniqueName: \"kubernetes.io/projected/5f155a88-e612-4fbd-a32e-906d0e74ce6c-kube-api-access-cmpb2\") pod \"multus-additional-cni-plugins-8cfrg\" (UID: \"5f155a88-e612-4fbd-a32e-906d0e74ce6c\") " pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.119072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.119013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"5d21ee4b15ed3fe9ce757cd4b6354bec","Type":"ContainerStarted","Data":"6058224295b182558040d9b88e1a6fe9b3577cbcb9cf9be45b54faf2baa5741c"}
Jul 28 23:21:42.121247 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.121219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv5rr\" (UniqueName: \"kubernetes.io/projected/3fcbd120-7554-4f46-a29c-1a32bb6155bc-kube-api-access-bv5rr\") pod \"iptables-alerter-mzxj8\" (UID: \"3fcbd120-7554-4f46-a29c-1a32bb6155bc\") " pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.121661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.121637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c96c4a66-6319-41f3-b325-25bbdbff0d52-agent-certs\") pod \"konnectivity-agent-4bvx6\" (UID: \"c96c4a66-6319-41f3-b325-25bbdbff0d52\") " pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.121844 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.121809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb8dr\" (UniqueName: \"kubernetes.io/projected/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-kube-api-access-wb8dr\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.124739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.124713    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gg5\" (UniqueName: \"kubernetes.io/projected/81c049c3-c1fd-411d-8631-18d11ec594a4-kube-api-access-q9gg5\") pod \"aws-ebs-csi-driver-node-gkxqg\" (UID: \"81c049c3-c1fd-411d-8631-18d11ec594a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.193954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.193912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7rxd"
Jul 28 23:21:42.202406 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.202369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zwzzw"
Jul 28 23:21:42.208033 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.208002    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0906a2_dc46_42c2_845a_a87c3d117a1b.slice/crio-66a57bc9c69ae2ecd6a3a81d58563282402fa35af6c2c73cb50be968b2a474d4 WatchSource:0}: Error finding container 66a57bc9c69ae2ecd6a3a81d58563282402fa35af6c2c73cb50be968b2a474d4: Status 404 returned error can't find the container with id 66a57bc9c69ae2ecd6a3a81d58563282402fa35af6c2c73cb50be968b2a474d4
Jul 28 23:21:42.215051 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.214880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h2ts6"
Jul 28 23:21:42.215051 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.214894    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cf8882_071f_4e89_9728_13511310389b.slice/crio-7769281de37e496df959b3415407d0b83b46de1c7f142b92a6e2b6d9080b9dc1 WatchSource:0}: Error finding container 7769281de37e496df959b3415407d0b83b46de1c7f142b92a6e2b6d9080b9dc1: Status 404 returned error can't find the container with id 7769281de37e496df959b3415407d0b83b46de1c7f142b92a6e2b6d9080b9dc1
Jul 28 23:21:42.222858 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.222665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:21:42.226067 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.225949    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7bac126_cd63_4eb0_9e70_77bd8be227f8.slice/crio-f9dca75034f402d3257979459d8e06570f05c4da7732b580d5703974b15faf46 WatchSource:0}: Error finding container f9dca75034f402d3257979459d8e06570f05c4da7732b580d5703974b15faf46: Status 404 returned error can't find the container with id f9dca75034f402d3257979459d8e06570f05c4da7732b580d5703974b15faf46
Jul 28 23:21:42.232207 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.232179    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:21:42.234177 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.234146    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083244cb_7b2a_4780_8636_faf88f1304da.slice/crio-23e4d45f6d4d1f1c48fbd3b4f801d89bd4e6a47fa30986b076cde40339f935e4 WatchSource:0}: Error finding container 23e4d45f6d4d1f1c48fbd3b4f801d89bd4e6a47fa30986b076cde40339f935e4: Status 404 returned error can't find the container with id 23e4d45f6d4d1f1c48fbd3b4f801d89bd4e6a47fa30986b076cde40339f935e4
Jul 28 23:21:42.240344 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.240123    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gwzhj"
Jul 28 23:21:42.248646 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.248620    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96c4a66_6319_41f3_b325_25bbdbff0d52.slice/crio-ef279fdaaf9e6fd8913abedd15d6a35ea45baab1b66f690da2a75571fe40cf58 WatchSource:0}: Error finding container ef279fdaaf9e6fd8913abedd15d6a35ea45baab1b66f690da2a75571fe40cf58: Status 404 returned error can't find the container with id ef279fdaaf9e6fd8913abedd15d6a35ea45baab1b66f690da2a75571fe40cf58
Jul 28 23:21:42.248751 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.248645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzxj8"
Jul 28 23:21:42.252787 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.252755    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddee4c9a_6fe1_4dca_a5fa_f5607136a2c3.slice/crio-2af6ba5dcb127bf43e2cb5fb5d63f3aeee69eceb36854717ef7b13d734d4b85d WatchSource:0}: Error finding container 2af6ba5dcb127bf43e2cb5fb5d63f3aeee69eceb36854717ef7b13d734d4b85d: Status 404 returned error can't find the container with id 2af6ba5dcb127bf43e2cb5fb5d63f3aeee69eceb36854717ef7b13d734d4b85d
Jul 28 23:21:42.258062 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.258008    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg"
Jul 28 23:21:42.259754 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.259730    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcbd120_7554_4f46_a29c_1a32bb6155bc.slice/crio-679f6738bd711422e6ffd2e95cbb505320ef6a48d8773cdb6049711d9862cea1 WatchSource:0}: Error finding container 679f6738bd711422e6ffd2e95cbb505320ef6a48d8773cdb6049711d9862cea1: Status 404 returned error can't find the container with id 679f6738bd711422e6ffd2e95cbb505320ef6a48d8773cdb6049711d9862cea1
Jul 28 23:21:42.265741 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.265718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8cfrg"
Jul 28 23:21:42.269368 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.269340    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c049c3_c1fd_411d_8631_18d11ec594a4.slice/crio-285034c32a2a40b8307400028fc5a896a2baec737e7769e6be3ef8c5e0ec51d3 WatchSource:0}: Error finding container 285034c32a2a40b8307400028fc5a896a2baec737e7769e6be3ef8c5e0ec51d3: Status 404 returned error can't find the container with id 285034c32a2a40b8307400028fc5a896a2baec737e7769e6be3ef8c5e0ec51d3
Jul 28 23:21:42.276049 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:21:42.276019    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f155a88_e612_4fbd_a32e_906d0e74ce6c.slice/crio-d95b0e7a7eab0b3168db32bc64ec804c99419bf2b0197e1a30bdb53ca7847613 WatchSource:0}: Error finding container d95b0e7a7eab0b3168db32bc64ec804c99419bf2b0197e1a30bdb53ca7847613: Status 404 returned error can't find the container with id d95b0e7a7eab0b3168db32bc64ec804c99419bf2b0197e1a30bdb53ca7847613
Jul 28 23:21:42.607399 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.607283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:42.607399 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.607335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607476    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607511    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607536    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607549    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:43.607535362 +0000 UTC m=+4.134584607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:42.607649 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.607604    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:43.607586471 +0000 UTC m=+4.134635717 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:42.708111 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.708073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:42.708284 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.708221    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:42.708360 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:42.708301    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:43.708280966 +0000 UTC m=+4.235330235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:42.932068 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.931959    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 23:16:42 +0000 UTC" deadline="2028-02-01 05:02:43.043774617 +0000 UTC"
Jul 28 23:21:42.932068 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:42.932005    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13253h41m0.111775354s"
Jul 28 23:21:43.001699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.001658    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:21:43.108740 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.108696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:43.108926 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.108834    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:43.109263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.109232    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:43.109346 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.109323    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:43.121921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.121889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gwzhj" event={"ID":"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3","Type":"ContainerStarted","Data":"2af6ba5dcb127bf43e2cb5fb5d63f3aeee69eceb36854717ef7b13d734d4b85d"}
Jul 28 23:21:43.123534 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.123506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h2ts6" event={"ID":"c7bac126-cd63-4eb0-9e70-77bd8be227f8","Type":"ContainerStarted","Data":"f9dca75034f402d3257979459d8e06570f05c4da7732b580d5703974b15faf46"}
Jul 28 23:21:43.125124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.125071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerStarted","Data":"d95b0e7a7eab0b3168db32bc64ec804c99419bf2b0197e1a30bdb53ca7847613"}
Jul 28 23:21:43.126524 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.126493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzxj8" event={"ID":"3fcbd120-7554-4f46-a29c-1a32bb6155bc","Type":"ContainerStarted","Data":"679f6738bd711422e6ffd2e95cbb505320ef6a48d8773cdb6049711d9862cea1"}
Jul 28 23:21:43.127841 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.127817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bvx6" event={"ID":"c96c4a66-6319-41f3-b325-25bbdbff0d52","Type":"ContainerStarted","Data":"ef279fdaaf9e6fd8913abedd15d6a35ea45baab1b66f690da2a75571fe40cf58"}
Jul 28 23:21:43.129394 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.129368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"23e4d45f6d4d1f1c48fbd3b4f801d89bd4e6a47fa30986b076cde40339f935e4"}
Jul 28 23:21:43.131233 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.131206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zwzzw" event={"ID":"78cf8882-071f-4e89-9728-13511310389b","Type":"ContainerStarted","Data":"7769281de37e496df959b3415407d0b83b46de1c7f142b92a6e2b6d9080b9dc1"}
Jul 28 23:21:43.132660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.132635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7rxd" event={"ID":"4b0906a2-dc46-42c2-845a-a87c3d117a1b","Type":"ContainerStarted","Data":"66a57bc9c69ae2ecd6a3a81d58563282402fa35af6c2c73cb50be968b2a474d4"}
Jul 28 23:21:43.134340 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.134310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg" event={"ID":"81c049c3-c1fd-411d-8631-18d11ec594a4","Type":"ContainerStarted","Data":"285034c32a2a40b8307400028fc5a896a2baec737e7769e6be3ef8c5e0ec51d3"}
Jul 28 23:21:43.615369 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.615302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:43.615369 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.615358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:43.615651 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.615632    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:43.615717 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.615658    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:43.615717 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.615672    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:43.615817 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.615732    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:45.615713278 +0000 UTC m=+6.142762535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:43.616210 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.616189    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:43.616316 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.616261    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:45.616244133 +0000 UTC m=+6.143293382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:43.716761 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.716719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:43.716931 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.716920    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:43.716995 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:43.716983    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:45.716963687 +0000 UTC m=+6.244012936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:43.932722 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.932592    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 23:16:42 +0000 UTC" deadline="2028-03-19 02:37:16.366644253 +0000 UTC"
Jul 28 23:21:43.932722 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:43.932638    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14379h15m32.434011438s"
Jul 28 23:21:44.111523 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:44.110850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:44.111523 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:44.111051    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:45.110024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.109647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:45.110690 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.110049    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:45.110690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.110133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:45.110690 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.110204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:45.148616 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.148529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"5d21ee4b15ed3fe9ce757cd4b6354bec","Type":"ContainerStarted","Data":"a0438d227d85d957d3e88c5a473ecb01a6208e593b2f54c7f89ffd4ff7ef09a7"}
Jul 28 23:21:45.162836 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.162336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" event={"ID":"39abfa21be29f5a114b5174162890ec9","Type":"ContainerStarted","Data":"9ac92ecf551654eee538147859e6b3f691a14b4588be8110a317d8980505a85d"}
Jul 28 23:21:45.639506 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.639467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:45.639712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.639521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:45.639775 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.639713    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:45.639775 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.639740    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:45.639775 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.639755    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:45.639922 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.639816    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:49.63979711 +0000 UTC m=+10.166846357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:45.640315 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.640266    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:45.640437 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.640373    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:49.640355336 +0000 UTC m=+10.167404593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:45.740665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:45.740629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:45.740832 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.740786    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:45.740891 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:45.740857    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:49.74083965 +0000 UTC m=+10.267888908 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:46.110895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:46.110810    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:46.111299 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:46.110959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:47.109820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:47.109314    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:47.109820 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:47.109436    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:47.109820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:47.109719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:47.109820 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:47.109778    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:48.109438 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:48.109338    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:48.109896 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:48.109493    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:48.171015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:48.170982    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d21ee4b15ed3fe9ce757cd4b6354bec" containerID="a0438d227d85d957d3e88c5a473ecb01a6208e593b2f54c7f89ffd4ff7ef09a7" exitCode=0
Jul 28 23:21:48.171185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:48.171037    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"5d21ee4b15ed3fe9ce757cd4b6354bec","Type":"ContainerDied","Data":"a0438d227d85d957d3e88c5a473ecb01a6208e593b2f54c7f89ffd4ff7ef09a7"}
Jul 28 23:21:48.190103 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:48.189942    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" podStartSLOduration=7.189919766 podStartE2EDuration="7.189919766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:21:45.179702075 +0000 UTC m=+5.706751362" watchObservedRunningTime="2026-07-28 23:21:48.189919766 +0000 UTC m=+8.716969029"
Jul 28 23:21:49.109707 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:49.109667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:49.110184 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:49.109721    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:49.110184 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.109819    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:49.110184 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.109960    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:49.681247 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:49.681212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:49.681444 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:49.681257    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:49.681444 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681407    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:49.681444 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681435    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:49.681612 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681454    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:49.681612 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681466    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:49.681612 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681486    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:57.681464399 +0000 UTC m=+18.208513643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:49.681612 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.681515    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:57.68149916 +0000 UTC m=+18.208548402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:49.781865 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:49.781823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:49.782048 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.782014    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:49.782120 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:49.782092    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:21:57.782073034 +0000 UTC m=+18.309122290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:50.110958 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:50.110877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:50.111429 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:50.111011    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:51.109254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:51.109170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:51.109254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:51.109209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:51.109546 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:51.109306    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:51.109546 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:51.109423    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:52.109569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:52.109532    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:52.110000 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:52.109674    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:53.109100 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:53.109054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:53.109260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:53.109063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:53.109260 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:53.109157    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:53.109260 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:53.109238    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:54.109782 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:54.109748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:54.110215 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:54.109872    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:55.109707 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:55.109668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:55.109707 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:55.109698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:55.110177 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:55.109828    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:55.110177 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:55.109919    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:56.109296 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:56.109265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:56.109468 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:56.109425    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:57.109249 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:57.109208    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:57.109249 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:57.109253    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:57.109765 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.109336    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:57.109765 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.109442    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:21:57.743026 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:57.742972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:57.743026 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:57.743028    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:57.743301 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743193    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:21:57.743301 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743209    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:21:57.743301 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743220    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:57.743301 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743280    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:13.743267008 +0000 UTC m=+34.270316250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:21:57.743520 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743477    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:57.743574 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.743553    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:13.743535344 +0000 UTC m=+34.270584607 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:21:57.844249 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:57.844203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:57.844449 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.844380    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:57.844502 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:57.844459    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:13.844441625 +0000 UTC m=+34.371490887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:21:58.109321 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:58.109242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:21:58.109786 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:58.109394    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:21:59.108662 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:59.108630    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:21:59.108841 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:59.108755    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:21:59.108841 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:21:59.108812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:21:59.108920 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:21:59.108906    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:00.109552 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:00.109518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:00.109990 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:00.109650    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:01.109386 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:01.109355    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:01.109386 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:01.109382    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:01.109518 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:01.109475    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:01.109625 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:01.109605    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:02.109633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.109301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:02.110485 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:02.109735    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:02.197858 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.197828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"5d21ee4b15ed3fe9ce757cd4b6354bec","Type":"ContainerStarted","Data":"e42f6c913f02c7356803bf17b391f6458c8c0db6fa2b2522800830715ce08ede"}
Jul 28 23:22:02.199298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.199273    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="2754fc7937078441020d6650c4c5da59bd001ae5d606e3cf6573f27ba64746dc" exitCode=0
Jul 28 23:22:02.199404 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.199359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"2754fc7937078441020d6650c4c5da59bd001ae5d606e3cf6573f27ba64746dc"}
Jul 28 23:22:02.200847 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.200814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4bvx6" event={"ID":"c96c4a66-6319-41f3-b325-25bbdbff0d52","Type":"ContainerStarted","Data":"898792f5a0f26a84bea36c261b7b43c472697e1473f10a1053c76cc63e554df0"}
Jul 28 23:22:02.204010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.203991    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:22:02.204391 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204373    2564 generic.go:356] "Generic (PLEG): container finished" podID="083244cb-7b2a-4780-8636-faf88f1304da" containerID="49f930cc56c657107161cd39023c2bcc2b658c37c223053a1ba8cccac8e3028a" exitCode=1
Jul 28 23:22:02.204491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"5e2cb348c5e8627e856eb7b10f049ddeaf2fb10ba1d47a78d9f8dca9378a291e"}
Jul 28 23:22:02.204548 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"e6e8aabb8c62a9485908d2aaf16837c15f4cc04139c4fab53173f9caa9d8149b"}
Jul 28 23:22:02.204603 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"bd4845e96abec85591eb4567a87e715f29b3377fb666ba5c2deafa5b3469e79e"}
Jul 28 23:22:02.204656 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerDied","Data":"49f930cc56c657107161cd39023c2bcc2b658c37c223053a1ba8cccac8e3028a"}
Jul 28 23:22:02.204709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.204674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"24e14748a1c8fed32192b42313d323580065709d528b8666a4a1257b8e8d370f"}
Jul 28 23:22:02.206174 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.206151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zwzzw" event={"ID":"78cf8882-071f-4e89-9728-13511310389b","Type":"ContainerStarted","Data":"7b91f52ab254b6533ac52dbe8168e9642a8d1df38243cc51d5acac44f36e0303"}
Jul 28 23:22:02.207594 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.207562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7rxd" event={"ID":"4b0906a2-dc46-42c2-845a-a87c3d117a1b","Type":"ContainerStarted","Data":"dfbf56997ac470d4309abfa6aa96873644e743a8f1553445e4042e10ada463aa"}
Jul 28 23:22:02.209248 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.209226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg" event={"ID":"81c049c3-c1fd-411d-8631-18d11ec594a4","Type":"ContainerStarted","Data":"32f515c628b74145af216d245f310b59be02765b6f2aa4ebf930f6bdce70a1fa"}
Jul 28 23:22:02.211321 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.211296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gwzhj" event={"ID":"ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3","Type":"ContainerStarted","Data":"15495faaacc84f73d3334004a54373e6e5bde1b35e0cf796206b51917586f49d"}
Jul 28 23:22:02.212055 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.212004    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" podStartSLOduration=21.21198874 podStartE2EDuration="21.21198874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.211935969 +0000 UTC m=+22.738985255" watchObservedRunningTime="2026-07-28 23:22:02.21198874 +0000 UTC m=+22.739038008"
Jul 28 23:22:02.213137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.213106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h2ts6" event={"ID":"c7bac126-cd63-4eb0-9e70-77bd8be227f8","Type":"ContainerStarted","Data":"35c6f4a32ac6b1b5628f989d6f1c001a4e38ed6c3755c60e4daefa86e399fe22"}
Jul 28 23:22:02.225981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.225927    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zwzzw" podStartSLOduration=3.239595558 podStartE2EDuration="22.225911002s" totalImagesPullingTime="18.986315444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.225282661 +0000 UTC m=+22.752331928" watchObservedRunningTime="2026-07-28 23:22:02.225911002 +0000 UTC m=+22.752960266"
Jul 28 23:22:02.257702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.257655    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gwzhj" podStartSLOduration=3.24315511 podStartE2EDuration="22.257639345s" totalImagesPullingTime="19.014484235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.257438288 +0000 UTC m=+22.784487553" watchObservedRunningTime="2026-07-28 23:22:02.257639345 +0000 UTC m=+22.784688608"
Jul 28 23:22:02.258145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.258104    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4bvx6" podStartSLOduration=3.277834653 podStartE2EDuration="22.258097331s" totalImagesPullingTime="18.980262678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.24180829 +0000 UTC m=+22.768857558" watchObservedRunningTime="2026-07-28 23:22:02.258097331 +0000 UTC m=+22.785146595"
Jul 28 23:22:02.292766 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.292701    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x7rxd" podStartSLOduration=3.271686127 podStartE2EDuration="22.292680181s" totalImagesPullingTime="19.020994054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.292431793 +0000 UTC m=+22.819481057" watchObservedRunningTime="2026-07-28 23:22:02.292680181 +0000 UTC m=+22.819729448"
Jul 28 23:22:02.306282 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.306226    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h2ts6" podStartSLOduration=3.303391625 podStartE2EDuration="22.306205651s" totalImagesPullingTime="19.002814026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:02.305442783 +0000 UTC m=+22.832492049" watchObservedRunningTime="2026-07-28 23:22:02.306205651 +0000 UTC m=+22.833254917"
Jul 28 23:22:02.664362 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.664321    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:22:02.665264 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.665241    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:22:02.844987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.844955    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 23:22:02.974686 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.974580    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T23:22:02.844972859Z","UUID":"c6e30f94-6a5b-432c-a871-6d776d7edc47","Handler":null,"Name":"","Endpoint":""}
Jul 28 23:22:02.976228 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.976203    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 23:22:02.976373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:02.976249    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 23:22:03.109601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.109517    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:03.109601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.109557    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:03.110200 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:03.109689    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:03.110200 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:03.109794    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:03.217301 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.217261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzxj8" event={"ID":"3fcbd120-7554-4f46-a29c-1a32bb6155bc","Type":"ContainerStarted","Data":"034690d20b5a029ff6dbf0e5a8ed87441ee03d42791787fef9bda796e309b20e"}
Jul 28 23:22:03.223702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.223679    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:22:03.224142 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.224114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"0ce3844ffcf1dbc455de1084cff20f1c0dcf6ee23ede45493a6f69aa45f9cbce"}
Jul 28 23:22:03.226020 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.225994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg" event={"ID":"81c049c3-c1fd-411d-8631-18d11ec594a4","Type":"ContainerStarted","Data":"17d29bd60f107664f05825445ff2489e22ebec4d2a1636736c45b0dbad6060eb"}
Jul 28 23:22:03.226793 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.226772    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:22:03.226941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.226921    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4bvx6"
Jul 28 23:22:03.232073 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:03.232024    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mzxj8" podStartSLOduration=4.262291809 podStartE2EDuration="23.232008886s" totalImagesPullingTime="18.969717077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:03.231820009 +0000 UTC m=+23.758869274" watchObservedRunningTime="2026-07-28 23:22:03.232008886 +0000 UTC m=+23.759058150"
Jul 28 23:22:04.109081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:04.109046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:04.109297 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:04.109177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:04.231231 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:04.231096    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg" event={"ID":"81c049c3-c1fd-411d-8631-18d11ec594a4","Type":"ContainerStarted","Data":"fc28726165f4df8273ff6eb9ba459d4ed7fcc202e6fba737a4364b4fe7f577fc"}
Jul 28 23:22:04.249147 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:04.249087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkxqg" podStartSLOduration=3.006781604 podStartE2EDuration="24.249065267s" totalImagesPullingTime="21.242283663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:04.248706852 +0000 UTC m=+24.775756116" watchObservedRunningTime="2026-07-28 23:22:04.249065267 +0000 UTC m=+24.776114531"
Jul 28 23:22:05.109433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:05.109390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:05.109433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:05.109425    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:05.109691 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:05.109537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:05.109749 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:05.109686    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:05.237466 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:05.237440    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:22:05.237896 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:05.237786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"4f00a15b8aa5db8709ba7dd012470223e11179398fb5610d2a5e122b5e3dfe4e"}
Jul 28 23:22:06.109338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:06.109168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:06.109491 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:06.109426    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:07.109497 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.109280    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:07.110162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.109305    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:07.110162 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:07.109583    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:07.110162 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:07.109642    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:07.243721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.243691    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="6f0daf8dd5512d3f4647ba0be04eb031793e15f2fc66cf082ddcdc1e87909cb8" exitCode=0
Jul 28 23:22:07.243891 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.243779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"6f0daf8dd5512d3f4647ba0be04eb031793e15f2fc66cf082ddcdc1e87909cb8"}
Jul 28 23:22:07.247213 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.247194    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:22:07.247570 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.247549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"c5ebe930e76a0dcb865e4ed13477589b3d721b1d287d87776d5dcb7deb702a4c"}
Jul 28 23:22:07.247853 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.247789    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:07.247912 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.247856    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:07.248001 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.247985    2564 scope.go:117] "RemoveContainer" containerID="49f930cc56c657107161cd39023c2bcc2b658c37c223053a1ba8cccac8e3028a"
Jul 28 23:22:07.265043 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:07.265022    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:08.109803 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.109769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:08.110215 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:08.109918    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:08.253054 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.253032    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:22:08.253401 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.253380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" event={"ID":"083244cb-7b2a-4780-8636-faf88f1304da","Type":"ContainerStarted","Data":"b932c49aa0041a1a7988604eb81ec305ccfd2d85cf0c60f989095c5f33086694"}
Jul 28 23:22:08.253569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.253540    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:08.268851 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.268826    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:08.285098 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.285053    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm" podStartSLOduration=8.972545765 podStartE2EDuration="28.285035608s" totalImagesPullingTime="19.312489843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:08.284787236 +0000 UTC m=+28.811836499" watchObservedRunningTime="2026-07-28 23:22:08.285035608 +0000 UTC m=+28.812084871"
Jul 28 23:22:08.484786 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.484751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f4k6f"]
Jul 28 23:22:08.484947 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.484928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:08.485077 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:08.485051    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:08.487684 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.487661    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djhbc"]
Jul 28 23:22:08.487808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.487772    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:08.487897 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:08.487873    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:08.488333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.488311    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bw89j"]
Jul 28 23:22:08.488497 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:08.488481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:08.488593 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:08.488577    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:09.257263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:09.257227    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="70900d0be9b169d5cd6d47c7c05c655f31c25c7c5d708b578fb03a9374fd03df" exitCode=0
Jul 28 23:22:09.257739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:09.257306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"70900d0be9b169d5cd6d47c7c05c655f31c25c7c5d708b578fb03a9374fd03df"}
Jul 28 23:22:10.109813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:10.109779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:10.109977 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:10.109884    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:10.109977 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:10.109953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:10.110070 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:10.110028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:10.110108 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:10.110051    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:10.110108 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:10.110088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:11.264126 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:11.263943    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="6eae0864ee91ed0288cbb8886556a2f7fbca7633b2688f24a25df6d9630f5720" exitCode=0
Jul 28 23:22:11.264552 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:11.264028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"6eae0864ee91ed0288cbb8886556a2f7fbca7633b2688f24a25df6d9630f5720"}
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:12.109335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:12.109359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:12.109474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:12.109479    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:12.109560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:12.109704 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:12.109635    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:13.764776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:13.764705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:13.764776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:13.764741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764841    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764853    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764868    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764877    2564 projected.go:196] Error preparing data for projected volume kube-api-access-8dc68 for pod openshift-network-diagnostics/network-check-target-djhbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764902    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret podName:afaeb2da-9369-4aea-b14f-baf82476d5d0 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.76488647 +0000 UTC m=+66.291935716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret") pod "global-pull-secret-syncer-f4k6f" (UID: "afaeb2da-9369-4aea-b14f-baf82476d5d0") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:13.765291 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.764918    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68 podName:2faf873f-645e-48ad-8b47-c3351ee402b7 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.76490731 +0000 UTC m=+66.291956551 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dc68" (UniqueName: "kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68") pod "network-check-target-djhbc" (UID: "2faf873f-645e-48ad-8b47-c3351ee402b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:13.865333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:13.865295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:13.865509 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.865483    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:13.865588 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:13.865574    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.865549015 +0000 UTC m=+66.392598259 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:14.109826 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.109790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:14.109989 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.109917    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-djhbc" podUID="2faf873f-645e-48ad-8b47-c3351ee402b7"
Jul 28 23:22:14.110412 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.110390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:14.110537 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.110487    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bw89j" podUID="43cc8c79-993f-463f-9b5e-8aaaa4a9d068"
Jul 28 23:22:14.110660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.110633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:14.110760 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.110742    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4k6f" podUID="afaeb2da-9369-4aea-b14f-baf82476d5d0"
Jul 28 23:22:14.242142 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.242113    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeReady"
Jul 28 23:22:14.242319 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.242284    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 23:22:14.272046 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.272013    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:22:14.277330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.277300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.279678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.279651    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 23:22:14.279823 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.279673    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 23:22:14.279823 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.279770    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 23:22:14.280092 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.280076    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gxdzj\""
Jul 28 23:22:14.285846 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.285827    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 23:22:14.286430 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.286404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:22:14.288955 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.288934    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6rt8g"]
Jul 28 23:22:14.292794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.292774    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zdkqg"]
Jul 28 23:22:14.292902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.292884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.295243 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.295224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 23:22:14.295243 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.295238    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7kmkd\""
Jul 28 23:22:14.295426 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.295255    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 23:22:14.296844 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.296824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.297461 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.297439    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6rt8g"]
Jul 28 23:22:14.299237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.299118    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dpgd4\""
Jul 28 23:22:14.299237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.299146    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 23:22:14.299487 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.299316    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 23:22:14.300687 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.300670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zdkqg"]
Jul 28 23:22:14.301039 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.300820    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 23:22:14.369252 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sddg6\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369645 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369645 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.369645 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.369577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.410788 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.410745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:14.470712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-tmp-dir\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sddg6\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.470836    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:14.470887 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.470854    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.470915    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:14.970893729 +0000 UTC m=+35.497942985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbh4\" (UniqueName: \"kubernetes.io/projected/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-kube-api-access-bsbh4\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.470981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2405c937-b80c-4c4c-ab91-bd9ee88da811-cert\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.471220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-config-volume\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.471699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2nqz\" (UniqueName: \"kubernetes.io/projected/2405c937-b80c-4c4c-ab91-bd9ee88da811-kube-api-access-z2nqz\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.471940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.471897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.472193 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.472167    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.475440 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.475416    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.475440 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.475430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.478855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.478833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sddg6\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.479285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.479264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.495067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.495039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:14.572124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-config-volume\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.572124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2nqz\" (UniqueName: \"kubernetes.io/projected/2405c937-b80c-4c4c-ab91-bd9ee88da811-kube-api-access-z2nqz\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.572373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-tmp-dir\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.572373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.572373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbh4\" (UniqueName: \"kubernetes.io/projected/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-kube-api-access-bsbh4\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.572373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2405c937-b80c-4c4c-ab91-bd9ee88da811-cert\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.572373 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.572369    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:14.572587 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.572441    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:15.072420646 +0000 UTC m=+35.599469901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:14.572743 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-tmp-dir\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.572938 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.572918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-config-volume\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.575068 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.575044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2405c937-b80c-4c4c-ab91-bd9ee88da811-cert\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.580809 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.580785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbh4\" (UniqueName: \"kubernetes.io/projected/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-kube-api-access-bsbh4\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:14.580909 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.580840    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2nqz\" (UniqueName: \"kubernetes.io/projected/2405c937-b80c-4c4c-ab91-bd9ee88da811-kube-api-access-z2nqz\") pod \"ingress-canary-zdkqg\" (UID: \"2405c937-b80c-4c4c-ab91-bd9ee88da811\") " pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.611029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.610992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zdkqg"
Jul 28 23:22:14.975727 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:14.975694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:14.976588 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.975849    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:14.976588 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.975871    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:14.976588 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:14.975947    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:15.97592544 +0000 UTC m=+36.502974690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:15.076795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:15.076758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:15.076980 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:15.076920    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:15.077050 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:15.076998    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:16.076978172 +0000 UTC m=+36.604027413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:15.414820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:15.414790    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zdkqg"]
Jul 28 23:22:15.418495 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:22:15.418468    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2405c937_b80c_4c4c_ab91_bd9ee88da811.slice/crio-07103d2c512891f64d57b4e711e457c08ae47f8b53d4cde969b4316a4a656011 WatchSource:0}: Error finding container 07103d2c512891f64d57b4e711e457c08ae47f8b53d4cde969b4316a4a656011: Status 404 returned error can't find the container with id 07103d2c512891f64d57b4e711e457c08ae47f8b53d4cde969b4316a4a656011
Jul 28 23:22:15.984437 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:15.984385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:15.984905 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:15.984538    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:15.984905 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:15.984555    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:15.984905 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:15.984627    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:17.984605462 +0000 UTC m=+38.511654709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:16.085719 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.085677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:16.085923 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:16.085855    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:16.085994 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:16.085933    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:18.085913336 +0000 UTC m=+38.612962591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:16.109473 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.109443    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:16.109627 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.109443    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:16.109681 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.109443    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:16.112187 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.112168    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bhxv7\""
Jul 28 23:22:16.113280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.113259    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plrk7\""
Jul 28 23:22:16.113417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.113281    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 23:22:16.113417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.113263    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 23:22:16.113417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.113281    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 23:22:16.113417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.113263    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 23:22:16.282942 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.282863    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="3fba48ee0a4f181ca444bea884c3a412f9829682bbf1bbe1107cdd085bf25925" exitCode=0
Jul 28 23:22:16.283100 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.282941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"3fba48ee0a4f181ca444bea884c3a412f9829682bbf1bbe1107cdd085bf25925"}
Jul 28 23:22:16.284389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:16.284096    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zdkqg" event={"ID":"2405c937-b80c-4c4c-ab91-bd9ee88da811","Type":"ContainerStarted","Data":"07103d2c512891f64d57b4e711e457c08ae47f8b53d4cde969b4316a4a656011"}
Jul 28 23:22:17.292497 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:17.292456    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f155a88-e612-4fbd-a32e-906d0e74ce6c" containerID="2b5bc30b20de0e6cf4f5f820c930a5b493bcf42af846349faf84bc3b29b09a6c" exitCode=0
Jul 28 23:22:17.292497 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:17.292508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerDied","Data":"2b5bc30b20de0e6cf4f5f820c930a5b493bcf42af846349faf84bc3b29b09a6c"}
Jul 28 23:22:18.000231 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.000197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:18.000425 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:18.000373    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:18.000425 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:18.000394    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:18.000522 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:18.000464    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:22.000444463 +0000 UTC m=+42.527493714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:18.101540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.101324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:18.101690 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:18.101495    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:18.101690 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:18.101613    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:22.101593188 +0000 UTC m=+42.628642430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:18.298246 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.298219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" event={"ID":"5f155a88-e612-4fbd-a32e-906d0e74ce6c","Type":"ContainerStarted","Data":"eac955ed1bcd57d91149635770bd403904a8b477814dea67a3d74761b3fa0343"}
Jul 28 23:22:18.326861 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.326817    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8cfrg" podStartSLOduration=9.077433064 podStartE2EDuration="38.326800385s" totalImagesPullingTime="29.249367321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:18.325472343 +0000 UTC m=+38.852521606" watchObservedRunningTime="2026-07-28 23:22:18.326800385 +0000 UTC m=+38.853849647"
Jul 28 23:22:18.451018 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.450987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:18.499940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.499903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:18.581855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.581819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:18.630483 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:18.630449    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:19.301780 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:19.301747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zdkqg" event={"ID":"2405c937-b80c-4c4c-ab91-bd9ee88da811","Type":"ContainerStarted","Data":"437d3397f852a1f7f3e5126cc50e2d9fc4656e9c596e786f732207b9335a9892"}
Jul 28 23:22:19.316689 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:19.316639    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zdkqg" podStartSLOduration=2.448535652 podStartE2EDuration="5.316623305s" totalImagesPullingTime="2.868087653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:19.315596028 +0000 UTC m=+39.842645292" watchObservedRunningTime="2026-07-28 23:22:19.316623305 +0000 UTC m=+39.843672567"
Jul 28 23:22:19.462721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:19.462668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:19.562086 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:19.561995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:20.741107 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:20.741073    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:20.842084 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:20.842043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:22.028846 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:22.028808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:22.029216 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:22.028967    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:22.029216 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:22.028987    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:22.029216 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:22.029041    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:30.029025046 +0000 UTC m=+50.556074287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:22.129890 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:22.129856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:22.130040 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:22.129996    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:22.130075 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:22.130057    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:30.130042243 +0000 UTC m=+50.657091486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:22.393558 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:22.393525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:22.494050 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:22.494002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:24.548663 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:24.548634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:24.865071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:24.864974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:26.149462 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:26.149425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:26.253476 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:26.253447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:26.349112 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:26.349078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:27.193363 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:27.193300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:27.245860 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:27.245808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:27.294903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:27.294869    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:30.090472 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:30.090439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:30.090847 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:30.090590    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:30.090847 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:30.090607    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:30.090847 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:30.090665    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:22:46.090647766 +0000 UTC m=+66.617697006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:30.191254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:30.191212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:30.191473 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:30.191370    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:30.191473 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:30.191461    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:46.191444985 +0000 UTC m=+66.718494229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:34.437357 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.437309    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"]
Jul 28 23:22:34.439249 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.439231    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.441681 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.441658    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:22:34.441795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.441659    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 23:22:34.442719 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.442702    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rhxfj\""
Jul 28 23:22:34.442813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.442735    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 23:22:34.450763 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.450740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"]
Jul 28 23:22:34.522833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.522792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thp4j\" (UniqueName: \"kubernetes.io/projected/69c84078-e201-4f4c-9da5-c7a0584afd41-kube-api-access-thp4j\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.523001 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.522903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.545753 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.545726    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8gkcs"]
Jul 28 23:22:34.547574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.547554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.549698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.549671    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l26sg"]
Jul 28 23:22:34.550679 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.550652    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 23:22:34.550785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.550735    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 23:22:34.551020 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.551004    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 23:22:34.551183 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.551167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xgcfq\""
Jul 28 23:22:34.551419 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.551395    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c5f6bbfdc-9q6tr"]
Jul 28 23:22:34.551547 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.551533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.551652 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.551631    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 23:22:34.554276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.554250    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 23:22:34.554405 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.554288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dqj97\""
Jul 28 23:22:34.554405 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.554365    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 23:22:34.554516 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.554490    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 23:22:34.556488 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.556468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.558572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.558553    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:22:34.562182 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.562148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.562656    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.562680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.562742    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.562656    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qdmh4\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.563014    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.563098    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.563106    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 23:22:34.563279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.563235    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 23:22:34.564079 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.564061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8gkcs"]
Jul 28 23:22:34.565423 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.564971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l26sg"]
Jul 28 23:22:34.566264 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.566247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5f6bbfdc-9q6tr"]
Jul 28 23:22:34.623949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.623919    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-snapshots\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.623949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.623956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.623949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.623984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-tmp\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.623949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f91cf43-36ec-47cb-870c-57d39b4b4de2-serving-cert\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.623949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thp4j\" (UniqueName: \"kubernetes.io/projected/69c84078-e201-4f4c-9da5-c7a0584afd41-kube-api-access-thp4j\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm6b7\" (UniqueName: \"kubernetes.io/projected/8f91cf43-36ec-47cb-870c-57d39b4b4de2-kube-api-access-pm6b7\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-stats-auth\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwj47\" (UniqueName: \"kubernetes.io/projected/f78be253-2630-4747-9957-79ad1db992af-kube-api-access-mwj47\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fef84116-f628-4535-a269-2e5bebfe6200-serving-cert\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-config\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.624421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f84n8\" (UniqueName: \"kubernetes.io/projected/fef84116-f628-4535-a269-2e5bebfe6200-kube-api-access-f84n8\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:34.624427    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-service-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:34.624549    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls podName:69c84078-e201-4f4c-9da5-c7a0584afd41 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:35.124527133 +0000 UTC m=+55.651576388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqwqm" (UID: "69c84078-e201-4f4c-9da5-c7a0584afd41") : secret "samples-operator-tls" not found
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-trusted-ca\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-default-certificate\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.624760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.624723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.634194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.634168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thp4j\" (UniqueName: \"kubernetes.io/projected/69c84078-e201-4f4c-9da5-c7a0584afd41-kube-api-access-thp4j\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:34.689532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.689455    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:34.725484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-trusted-ca\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.725591 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-default-certificate\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.725591 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.725693 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.725746 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-snapshots\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.725994 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:34.725960    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:35.225906551 +0000 UTC m=+55.752955796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:34.725994 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.725991    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-tmp\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f91cf43-36ec-47cb-870c-57d39b4b4de2-serving-cert\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm6b7\" (UniqueName: \"kubernetes.io/projected/8f91cf43-36ec-47cb-870c-57d39b4b4de2-kube-api-access-pm6b7\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726126    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-stats-auth\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwj47\" (UniqueName: \"kubernetes.io/projected/f78be253-2630-4747-9957-79ad1db992af-kube-api-access-mwj47\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:34.726159    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 23:22:34.726194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fef84116-f628-4535-a269-2e5bebfe6200-serving-cert\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.726574 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:34.726228    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:35.226209607 +0000 UTC m=+55.753258850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : secret "router-metrics-certs-default" not found
Jul 28 23:22:34.726574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-snapshots\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726761 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f91cf43-36ec-47cb-870c-57d39b4b4de2-tmp\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726846 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.726903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-trusted-ca\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.726903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-config\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.726981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f84n8\" (UniqueName: \"kubernetes.io/projected/fef84116-f628-4535-a269-2e5bebfe6200-kube-api-access-f84n8\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.727036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.726982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-service-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.727526 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.727505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fef84116-f628-4535-a269-2e5bebfe6200-config\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.727604 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.727526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f91cf43-36ec-47cb-870c-57d39b4b4de2-service-ca-bundle\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.728601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.728583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-default-certificate\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.728885 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.728862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fef84116-f628-4535-a269-2e5bebfe6200-serving-cert\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.729055 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.729038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f91cf43-36ec-47cb-870c-57d39b4b4de2-serving-cert\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.729232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.729216    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-stats-auth\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.734984 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.734963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm6b7\" (UniqueName: \"kubernetes.io/projected/8f91cf43-36ec-47cb-870c-57d39b4b4de2-kube-api-access-pm6b7\") pod \"insights-operator-5885fdb759-8gkcs\" (UID: \"8f91cf43-36ec-47cb-870c-57d39b4b4de2\") " pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.735195 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.735174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f84n8\" (UniqueName: \"kubernetes.io/projected/fef84116-f628-4535-a269-2e5bebfe6200-kube-api-access-f84n8\") pod \"console-operator-584dcb6b54-l26sg\" (UID: \"fef84116-f628-4535-a269-2e5bebfe6200\") " pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.735975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.735954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwj47\" (UniqueName: \"kubernetes.io/projected/f78be253-2630-4747-9957-79ad1db992af-kube-api-access-mwj47\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:34.766278 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.766245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:34.823792 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.823758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:34.860902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.860875    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-8gkcs"
Jul 28 23:22:34.869358 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.869316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:34.880155 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.880123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:34.991893 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:34.991864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-8gkcs"]
Jul 28 23:22:34.994710 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:22:34.994680    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f91cf43_36ec_47cb_870c_57d39b4b4de2.slice/crio-3641b8d19791a6b84caaa9c0b457fd0909655201c92ca0ee6ac4c766b52e5ceb WatchSource:0}: Error finding container 3641b8d19791a6b84caaa9c0b457fd0909655201c92ca0ee6ac4c766b52e5ceb: Status 404 returned error can't find the container with id 3641b8d19791a6b84caaa9c0b457fd0909655201c92ca0ee6ac4c766b52e5ceb
Jul 28 23:22:35.008926 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.008895    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l26sg"]
Jul 28 23:22:35.012851 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:22:35.012828    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef84116_f628_4535_a269_2e5bebfe6200.slice/crio-5f8508d89613d5de4a2f81bb6efa1080db8e9eef79fe08805056aaadaaa0909e WatchSource:0}: Error finding container 5f8508d89613d5de4a2f81bb6efa1080db8e9eef79fe08805056aaadaaa0909e: Status 404 returned error can't find the container with id 5f8508d89613d5de4a2f81bb6efa1080db8e9eef79fe08805056aaadaaa0909e
Jul 28 23:22:35.130567 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.130523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:35.130738 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:35.130644    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 23:22:35.130738 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:35.130697    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls podName:69c84078-e201-4f4c-9da5-c7a0584afd41 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.130684141 +0000 UTC m=+56.657733381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqwqm" (UID: "69c84078-e201-4f4c-9da5-c7a0584afd41") : secret "samples-operator-tls" not found
Jul 28 23:22:35.231203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.231124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:35.231203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.231166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:35.231383 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:35.231275    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 23:22:35.231383 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:35.231288    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.231269652 +0000 UTC m=+56.758318901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:35.231383 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:35.231309    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.231299936 +0000 UTC m=+56.758349177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : secret "router-metrics-certs-default" not found
Jul 28 23:22:35.338411 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.338374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" event={"ID":"fef84116-f628-4535-a269-2e5bebfe6200","Type":"ContainerStarted","Data":"5f8508d89613d5de4a2f81bb6efa1080db8e9eef79fe08805056aaadaaa0909e"}
Jul 28 23:22:35.339320 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:35.339295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8gkcs" event={"ID":"8f91cf43-36ec-47cb-870c-57d39b4b4de2","Type":"ContainerStarted","Data":"3641b8d19791a6b84caaa9c0b457fd0909655201c92ca0ee6ac4c766b52e5ceb"}
Jul 28 23:22:36.141211 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:36.141135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:36.141837 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:36.141249    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 23:22:36.141837 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:36.141359    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls podName:69c84078-e201-4f4c-9da5-c7a0584afd41 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.141322878 +0000 UTC m=+58.668372133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqwqm" (UID: "69c84078-e201-4f4c-9da5-c7a0584afd41") : secret "samples-operator-tls" not found
Jul 28 23:22:36.242057 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:36.241997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:36.242230 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:36.242164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:36.242314 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:36.242224    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.242201046 +0000 UTC m=+58.769250304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:36.242314 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:36.242294    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 23:22:36.242421 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:36.242374    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.242357683 +0000 UTC m=+58.769406939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : secret "router-metrics-certs-default" not found
Jul 28 23:22:37.396031 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:37.396005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:37.779184 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:37.779100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:37.831679 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:37.831635    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:38.159850 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.159799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:38.160026 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.159944    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 23:22:38.160026 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.160006    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls podName:69c84078-e201-4f4c-9da5-c7a0584afd41 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:42.159992506 +0000 UTC m=+62.687041752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqwqm" (UID: "69c84078-e201-4f4c-9da5-c7a0584afd41") : secret "samples-operator-tls" not found
Jul 28 23:22:38.261713 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.261679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:38.261883 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.261722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:38.261883 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.261868    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 23:22:38.262020 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.261914    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:42.261890004 +0000 UTC m=+62.788939266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:38.262020 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.261950    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:42.261935893 +0000 UTC m=+62.788985143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : secret "router-metrics-certs-default" not found
Jul 28 23:22:38.269769 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.269711    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nmcnl"]
Jul 28 23:22:38.273037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.273013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.275499 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.275477    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 23:22:38.275629 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.275588    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xh4g8\""
Jul 28 23:22:38.275696 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.275624    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 23:22:38.285617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.285595    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmcnl"]
Jul 28 23:22:38.308917 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.308895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:38.347616 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.347592    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/0.log"
Jul 28 23:22:38.347779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.347634    2564 generic.go:356] "Generic (PLEG): container finished" podID="fef84116-f628-4535-a269-2e5bebfe6200" containerID="5e02d93176a4135e96acb6e9eadaa184d8515a0738e08c120a01019a2dbfe59f" exitCode=255
Jul 28 23:22:38.347779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.347711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" event={"ID":"fef84116-f628-4535-a269-2e5bebfe6200","Type":"ContainerDied","Data":"5e02d93176a4135e96acb6e9eadaa184d8515a0738e08c120a01019a2dbfe59f"}
Jul 28 23:22:38.347976 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.347956    2564 scope.go:117] "RemoveContainer" containerID="5e02d93176a4135e96acb6e9eadaa184d8515a0738e08c120a01019a2dbfe59f"
Jul 28 23:22:38.349294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.349273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8gkcs" event={"ID":"8f91cf43-36ec-47cb-870c-57d39b4b4de2","Type":"ContainerStarted","Data":"c048668ebca4490146a36467904439e34fca5cdd617a60425d9fc9ed24e174d2"}
Jul 28 23:22:38.362771 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.362746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.362878 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.362819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9a684e9-2655-488b-bfc4-f8a7d89359f6-data-volume\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.362941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.362888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9a684e9-2655-488b-bfc4-f8a7d89359f6-crio-socket\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.363045 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.362993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mf46\" (UniqueName: \"kubernetes.io/projected/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-api-access-6mf46\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.363045 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.363028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.370864 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.370829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:38.380416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.380372    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-8gkcs" podStartSLOduration=2.000028992 podStartE2EDuration="4.380339758s" totalImagesPullingTime="2.380310766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:38.379594264 +0000 UTC m=+58.906643528" watchObservedRunningTime="2026-07-28 23:22:38.380339758 +0000 UTC m=+58.907389020"
Jul 28 23:22:38.464018 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.463995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mf46\" (UniqueName: \"kubernetes.io/projected/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-api-access-6mf46\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9a684e9-2655-488b-bfc4-f8a7d89359f6-data-volume\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464333 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9a684e9-2655-488b-bfc4-f8a7d89359f6-crio-socket\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464333 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.464259    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.464585 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.464338    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.96431814 +0000 UTC m=+59.491367410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.464585 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.464261    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:38.464585 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.464486    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.964467747 +0000 UTC m=+59.491517004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:38.464585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9a684e9-2655-488b-bfc4-f8a7d89359f6-data-volume\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.464851 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.464725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9a684e9-2655-488b-bfc4-f8a7d89359f6-crio-socket\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.471997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.471970    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mf46\" (UniqueName: \"kubernetes.io/projected/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-api-access-6mf46\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.969235 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.969191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.969422 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:38.969290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:38.969422 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.969320    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.969422 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.969407    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:38.969422 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.969413    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:39.969398614 +0000 UTC m=+60.496447858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.969592 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:38.969457    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:39.969445056 +0000 UTC m=+60.496494298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:39.353652 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.353623    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/1.log"
Jul 28 23:22:39.354040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.354025    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/0.log"
Jul 28 23:22:39.354092 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.354059    2564 generic.go:356] "Generic (PLEG): container finished" podID="fef84116-f628-4535-a269-2e5bebfe6200" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd" exitCode=255
Jul 28 23:22:39.354151 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.354137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" event={"ID":"fef84116-f628-4535-a269-2e5bebfe6200","Type":"ContainerDied","Data":"ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"}
Jul 28 23:22:39.354223 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.354211    2564 scope.go:117] "RemoveContainer" containerID="5e02d93176a4135e96acb6e9eadaa184d8515a0738e08c120a01019a2dbfe59f"
Jul 28 23:22:39.354469 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.354451    2564 scope.go:117] "RemoveContainer" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"
Jul 28 23:22:39.354667 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:39.354648    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l26sg_openshift-console-operator(fef84116-f628-4535-a269-2e5bebfe6200)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podUID="fef84116-f628-4535-a269-2e5bebfe6200"
Jul 28 23:22:39.941616 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.941582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:39.978124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.978099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:39.978276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:39.978192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:39.978276 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:39.978256    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:39.978362 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:39.978312    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:41.978297012 +0000 UTC m=+62.505346257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:39.978362 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:39.978263    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:39.978433 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:39.978386    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:41.978374532 +0000 UTC m=+62.505423773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:40.042452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.042399    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.149297 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.149252    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.211422 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.211306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.277934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.277903    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44zbm"
Jul 28 23:22:40.308410 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.308373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.358437 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.358415    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/1.log"
Jul 28 23:22:40.358764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.358748    2564 scope.go:117] "RemoveContainer" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"
Jul 28 23:22:40.358913 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:40.358894    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l26sg_openshift-console-operator(fef84116-f628-4535-a269-2e5bebfe6200)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podUID="fef84116-f628-4535-a269-2e5bebfe6200"
Jul 28 23:22:40.427412 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.427372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.464294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.464229    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zwzzw_78cf8882-071f-4e89-9728-13511310389b/dns-node-resolver/0.log"
Jul 28 23:22:40.478926 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.478895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.650085 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.650052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:40.750776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:40.750699    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:41.261901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:41.261875    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h2ts6_c7bac126-cd63-4eb0-9e70-77bd8be227f8/node-ca/0.log"
Jul 28 23:22:41.995441 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:41.995393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:41.995632 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:41.995516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:41.995632 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:41.995550    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:41.995632 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:41.995618    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:41.995770 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:41.995620    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.995603378 +0000 UTC m=+66.522652622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:41.995770 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:41.995682    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.995665895 +0000 UTC m=+66.522715143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:42.196809 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.196770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:42.196967 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:42.196896    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 23:22:42.196967 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:42.196949    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls podName:69c84078-e201-4f4c-9da5-c7a0584afd41 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:50.196934091 +0000 UTC m=+70.723983332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls") pod "cluster-samples-operator-6748489656-cqwqm" (UID: "69c84078-e201-4f4c-9da5-c7a0584afd41") : secret "samples-operator-tls" not found
Jul 28 23:22:42.244440 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.244406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:42.264585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.264510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zdkqg_2405c937-b80c-4c4c-ab91-bd9ee88da811/serve-healthcheck-canary/0.log"
Jul 28 23:22:42.297831 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.297793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:42.297973 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.297839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:42.297973 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:42.297942    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 23:22:42.298046 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:42.297980    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:50.297960404 +0000 UTC m=+70.825009644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:42.298046 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:42.298005    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:22:50.297996705 +0000 UTC m=+70.825045946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : secret "router-metrics-certs-default" not found
Jul 28 23:22:42.340684 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.340649    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:42.441491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.441454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:42.909597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:42.909564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:44.869604 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:44.869560    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:44.870005 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:44.869621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:22:44.870005 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:44.869948    2564 scope.go:117] "RemoveContainer" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"
Jul 28 23:22:44.870115 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:44.870097    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l26sg_openshift-console-operator(fef84116-f628-4535-a269-2e5bebfe6200)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podUID="fef84116-f628-4535-a269-2e5bebfe6200"
Jul 28 23:22:45.107714 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.107682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:45.499560 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.499527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:45.809356 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.809305    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:45.826747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.826723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:45.826881 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.826751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:45.829385 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.829369    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 23:22:45.829492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.829474    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 23:22:45.839569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.839546    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 23:22:45.839674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.839635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afaeb2da-9369-4aea-b14f-baf82476d5d0-original-pull-secret\") pod \"global-pull-secret-syncer-f4k6f\" (UID: \"afaeb2da-9369-4aea-b14f-baf82476d5d0\") " pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:45.849889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.849857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dc68\" (UniqueName: \"kubernetes.io/projected/2faf873f-645e-48ad-8b47-c3351ee402b7-kube-api-access-8dc68\") pod \"network-check-target-djhbc\" (UID: \"2faf873f-645e-48ad-8b47-c3351ee402b7\") " pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:45.861161 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.861140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:45.927541 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.927511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:22:45.930435 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:45.930417    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 23:22:45.938096 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:45.938072    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 23:22:45.938210 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:45.938134    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs podName:43cc8c79-993f-463f-9b5e-8aaaa4a9d068 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:49.93811794 +0000 UTC m=+130.465167181 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs") pod "network-metrics-daemon-bw89j" (UID: "43cc8c79-993f-463f-9b5e-8aaaa4a9d068") : secret "metrics-daemon-secret" not found
Jul 28 23:22:46.017997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.017932    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:46.028558 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.028524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:46.028891 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.028859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:46.029131 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.028614    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:46.029292 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.029280    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:54.029252943 +0000 UTC m=+74.556302199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:46.029427 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.029115    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:46.029555 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.029545    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:54.029526714 +0000 UTC m=+74.556575959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:46.121484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.121401    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bhxv7\""
Jul 28 23:22:46.122950 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.122923    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:46.128771 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.128752    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4k6f"
Jul 28 23:22:46.129857 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.129842    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:46.130070 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.130049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:22:46.130199 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.130183    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 23:22:46.130262 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.130200    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddc78c9cb-6mgmp: secret "image-registry-tls" not found
Jul 28 23:22:46.130262 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.130256    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls podName:fddade49-3504-4bdd-9164-cfd866172a4c nodeName:}" failed. No retries permitted until 2026-07-28 23:23:18.1302417 +0000 UTC m=+98.657290940 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls") pod "image-registry-ddc78c9cb-6mgmp" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c") : secret "image-registry-tls" not found
Jul 28 23:22:46.220626 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.220597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:46.230789 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.230751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:22:46.230930 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.230911    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:46.231007 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:46.230987    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls podName:d4cd0759-0e50-4f91-82b7-2d12fc26c4c9 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:18.230968148 +0000 UTC m=+98.758017390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls") pod "dns-default-6rt8g" (UID: "d4cd0759-0e50-4f91-82b7-2d12fc26c4c9") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:46.262179 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.262145    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f4k6f"]
Jul 28 23:22:46.265697 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:22:46.265667    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafaeb2da_9369_4aea_b14f_baf82476d5d0.slice/crio-d1b016051ceab850482a43d4e27ecddc9a46856315147efd1bdef3a5d3bdee3e WatchSource:0}: Error finding container d1b016051ceab850482a43d4e27ecddc9a46856315147efd1bdef3a5d3bdee3e: Status 404 returned error can't find the container with id d1b016051ceab850482a43d4e27ecddc9a46856315147efd1bdef3a5d3bdee3e
Jul 28 23:22:46.277076 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.277043    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-djhbc"]
Jul 28 23:22:46.280501 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:22:46.280475    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2faf873f_645e_48ad_8b47_c3351ee402b7.slice/crio-0d44201844090e412482c637d1a206ecb63ec33683c3aa5260728092b475c541 WatchSource:0}: Error finding container 0d44201844090e412482c637d1a206ecb63ec33683c3aa5260728092b475c541: Status 404 returned error can't find the container with id 0d44201844090e412482c637d1a206ecb63ec33683c3aa5260728092b475c541
Jul 28 23:22:46.375031 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.374941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djhbc" event={"ID":"2faf873f-645e-48ad-8b47-c3351ee402b7","Type":"ContainerStarted","Data":"0d44201844090e412482c637d1a206ecb63ec33683c3aa5260728092b475c541"}
Jul 28 23:22:46.376003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:46.375978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f4k6f" event={"ID":"afaeb2da-9369-4aea-b14f-baf82476d5d0","Type":"ContainerStarted","Data":"d1b016051ceab850482a43d4e27ecddc9a46856315147efd1bdef3a5d3bdee3e"}
Jul 28 23:22:47.005386 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:47.005330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:47.596039 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:47.595991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:47.800240 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:47.800202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:48.196754 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:48.196725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:48.396093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:48.396037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:48.596541 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:48.596504    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:49.198278 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:49.198240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:49.397798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:49.397761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:49.594540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:49.594509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.001955 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.001864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.193052 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.193010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.243137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.243106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.267187 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.267108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:50.269840 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.269815    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/69c84078-e201-4f4c-9da5-c7a0584afd41-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-cqwqm\" (UID: \"69c84078-e201-4f4c-9da5-c7a0584afd41\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:50.350855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.350824    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rhxfj\""
Jul 28 23:22:50.358672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.358646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"
Jul 28 23:22:50.368566 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.368532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:50.368651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.368586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:50.368714 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:50.368697    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle podName:f78be253-2630-4747-9957-79ad1db992af nodeName:}" failed. No retries permitted until 2026-07-28 23:23:06.368677219 +0000 UTC m=+86.895726461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle") pod "router-default-c5f6bbfdc-9q6tr" (UID: "f78be253-2630-4747-9957-79ad1db992af") : configmap references non-existent config key: service-ca.crt
Jul 28 23:22:50.370965 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.370940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78be253-2630-4747-9957-79ad1db992af-metrics-certs\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:22:50.393876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.393840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.800580 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.800520    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:50.869109 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:50.869075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm"]
Jul 28 23:22:51.392117 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.392078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-djhbc" event={"ID":"2faf873f-645e-48ad-8b47-c3351ee402b7","Type":"ContainerStarted","Data":"337627e8698d08704a7ed6ee47883fa5201887d3101e6913d8d4eb4c18ef6937"}
Jul 28 23:22:51.392556 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.392165    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:22:51.393655 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.393620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f4k6f" event={"ID":"afaeb2da-9369-4aea-b14f-baf82476d5d0","Type":"ContainerStarted","Data":"06107c5f8bfb9b7a0da672b5e9515f1bdd0776b7a9b0c4bad5f3ab944b6bd202"}
Jul 28 23:22:51.394639 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.394618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm" event={"ID":"69c84078-e201-4f4c-9da5-c7a0584afd41","Type":"ContainerStarted","Data":"87dda3461b225e2609cff6e79ea3c95fa5657b1ce749eff014e4eec078115afa"}
Jul 28 23:22:51.405611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.405564    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-djhbc" podStartSLOduration=66.942031316 podStartE2EDuration="1m11.405551077s" totalImagesPullingTime="4.463519761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:51.405441324 +0000 UTC m=+71.932490589" watchObservedRunningTime="2026-07-28 23:22:51.405551077 +0000 UTC m=+71.932600340"
Jul 28 23:22:51.418993 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.418934    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f4k6f" podStartSLOduration=66.937191187 podStartE2EDuration="1m11.418920296s" totalImagesPullingTime="4.481729109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:51.418691994 +0000 UTC m=+71.945741256" watchObservedRunningTime="2026-07-28 23:22:51.418920296 +0000 UTC m=+71.945969559"
Jul 28 23:22:51.439670 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.439631    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:51.594738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.594700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:51.782906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.782818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:51.997638 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:51.997607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:52.198951 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:52.198917    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:52.400592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:52.400523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:52.826358 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:52.826265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:53.398576 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:53.398543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:53.404577 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:53.404549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm" event={"ID":"69c84078-e201-4f4c-9da5-c7a0584afd41","Type":"ContainerStarted","Data":"9337e9246e4e7a72ef459b011f9ed2568352c49ee2dc8a187be56dfd8d31a2ee"}
Jul 28 23:22:53.404947 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:53.404598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm" event={"ID":"69c84078-e201-4f4c-9da5-c7a0584afd41","Type":"ContainerStarted","Data":"2b4f218398f757577d6cabc30e0099ecf21a8c1037ad4e9ef032b2cc94f931c9"}
Jul 28 23:22:53.419304 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:53.419253    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-cqwqm" podStartSLOduration=17.778024874 podStartE2EDuration="19.419235215s" totalImagesPullingTime="1.641210341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:53.418469838 +0000 UTC m=+73.945519102" watchObservedRunningTime="2026-07-28 23:22:53.419235215 +0000 UTC m=+73.946284482"
Jul 28 23:22:53.797271 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:53.797245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:54.099452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.099340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:54.099452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.099438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:54.099636 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:54.099540    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:54.099636 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:54.099617    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls podName:d9a684e9-2655-488b-bfc4-f8a7d89359f6 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:10.099597602 +0000 UTC m=+90.626646846 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmcnl" (UID: "d9a684e9-2655-488b-bfc4-f8a7d89359f6") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:54.099894 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.099878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9a684e9-2655-488b-bfc4-f8a7d89359f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:22:54.194420 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.194390    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:54.426958 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.426876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:54.597391 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.597361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:54.802156 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.802119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:54.996624 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:54.996587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:55.605783 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:55.605749    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:55.996883 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:55.996786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:56.196680 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:56.196633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:22:57.108954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.108924    2564 scope.go:117] "RemoveContainer" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"
Jul 28 23:22:57.418258 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.418182    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:22:57.418641 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.418626    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/1.log"
Jul 28 23:22:57.418703 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.418658    2564 generic.go:356] "Generic (PLEG): container finished" podID="fef84116-f628-4535-a269-2e5bebfe6200" containerID="dd33d6c03f7313b2f137b5e467cc005f9cf87cb90dc5926904ae576f85ccc531" exitCode=255
Jul 28 23:22:57.418755 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.418707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" event={"ID":"fef84116-f628-4535-a269-2e5bebfe6200","Type":"ContainerDied","Data":"dd33d6c03f7313b2f137b5e467cc005f9cf87cb90dc5926904ae576f85ccc531"}
Jul 28 23:22:57.418755 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.418750    2564 scope.go:117] "RemoveContainer" containerID="ae6f0798793f762da2aa1508913b3809f0538e6adc0e37edd7aed7fa4e61b8bd"
Jul 28 23:22:57.419105 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:57.419081    2564 scope.go:117] "RemoveContainer" containerID="dd33d6c03f7313b2f137b5e467cc005f9cf87cb90dc5926904ae576f85ccc531"
Jul 28 23:22:57.419304 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:22:57.419275    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-l26sg_openshift-console-operator(fef84116-f628-4535-a269-2e5bebfe6200)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podUID="fef84116-f628-4535-a269-2e5bebfe6200"
Jul 28 23:22:58.423199 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:22:58.423169    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:23:01.729919 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:01.729886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:01.800968 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:01.800934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:01.907134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:01.907096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:04.224466 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:04.224329    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:04.324437 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:04.324398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:04.870419 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:04.870381    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:23:04.870666 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:04.870448    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:23:04.870851 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:04.870834    2564 scope.go:117] "RemoveContainer" containerID="dd33d6c03f7313b2f137b5e467cc005f9cf87cb90dc5926904ae576f85ccc531"
Jul 28 23:23:04.871049 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:04.871028    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-l26sg_openshift-console-operator(fef84116-f628-4535-a269-2e5bebfe6200)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podUID="fef84116-f628-4535-a269-2e5bebfe6200"
Jul 28 23:23:05.740018 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:05.739967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:05.791005 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:05.790973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:06.400116 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:06.400079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:06.400699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:06.400681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78be253-2630-4747-9957-79ad1db992af-service-ca-bundle\") pod \"router-default-c5f6bbfdc-9q6tr\" (UID: \"f78be253-2630-4747-9957-79ad1db992af\") " pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:06.676467 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:06.676388    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qdmh4\""
Jul 28 23:23:06.684460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:06.684438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:06.809607 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:06.809574    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5f6bbfdc-9q6tr"]
Jul 28 23:23:06.813320 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:06.813293    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78be253_2630_4747_9957_79ad1db992af.slice/crio-183f59621f5dbfedde68a79a19694ee5bbca98b1ee334a4cef4ced52098bd9ca WatchSource:0}: Error finding container 183f59621f5dbfedde68a79a19694ee5bbca98b1ee334a4cef4ced52098bd9ca: Status 404 returned error can't find the container with id 183f59621f5dbfedde68a79a19694ee5bbca98b1ee334a4cef4ced52098bd9ca
Jul 28 23:23:07.452903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:07.452869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr" event={"ID":"f78be253-2630-4747-9957-79ad1db992af","Type":"ContainerStarted","Data":"c1dc81ad339482ffb507a24c0f6b7a285c2d3429284539ddc4e1724d06ea847b"}
Jul 28 23:23:07.453115 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:07.452916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr" event={"ID":"f78be253-2630-4747-9957-79ad1db992af","Type":"ContainerStarted","Data":"183f59621f5dbfedde68a79a19694ee5bbca98b1ee334a4cef4ced52098bd9ca"}
Jul 28 23:23:07.473591 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:07.473536    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr" podStartSLOduration=33.473518316 podStartE2EDuration="33.473518316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:07.472325077 +0000 UTC m=+87.999374341" watchObservedRunningTime="2026-07-28 23:23:07.473518316 +0000 UTC m=+88.000567634"
Jul 28 23:23:07.684988 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:07.684951    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:07.687439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:07.687414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:08.456718 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:08.456679    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:08.457938 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:08.457913    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c5f6bbfdc-9q6tr"
Jul 28 23:23:09.108356 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.108324    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.212056 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.212025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.268425 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.268394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.271507 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.271482    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"]
Jul 28 23:23:09.274814 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.274799    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:09.277189 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.277162    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 23:23:09.277436 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.277419    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n8g9l\""
Jul 28 23:23:09.281416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.281393    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"]
Jul 28 23:23:09.324472 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.324430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xs845\" (UID: \"ec64a69f-f83c-4799-9452-6afc6af4e254\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:09.424748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.424672    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xs845\" (UID: \"ec64a69f-f83c-4799-9452-6afc6af4e254\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:09.424888 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:09.424829    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 23:23:09.424933 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:09.424909    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates podName:ec64a69f-f83c-4799-9452-6afc6af4e254 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:09.924891091 +0000 UTC m=+90.451940335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-xs845" (UID: "ec64a69f-f83c-4799-9452-6afc6af4e254") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 23:23:09.533322 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.533289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.591393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.591360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.680146 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.680064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:09.929665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.929628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xs845\" (UID: \"ec64a69f-f83c-4799-9452-6afc6af4e254\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:09.932031 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:09.931974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ec64a69f-f83c-4799-9452-6afc6af4e254-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xs845\" (UID: \"ec64a69f-f83c-4799-9452-6afc6af4e254\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:10.130302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.130270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:23:10.132550 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.132528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9a684e9-2655-488b-bfc4-f8a7d89359f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmcnl\" (UID: \"d9a684e9-2655-488b-bfc4-f8a7d89359f6\") " pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:23:10.183633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.183533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:10.308944 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.308910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"]
Jul 28 23:23:10.311801 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:10.311769    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec64a69f_f83c_4799_9452_6afc6af4e254.slice/crio-912528c3728cc81c49b0df0b8e52d024bd722533aa9eeb9989bd2a91cca38ebf WatchSource:0}: Error finding container 912528c3728cc81c49b0df0b8e52d024bd722533aa9eeb9989bd2a91cca38ebf: Status 404 returned error can't find the container with id 912528c3728cc81c49b0df0b8e52d024bd722533aa9eeb9989bd2a91cca38ebf
Jul 28 23:23:10.384144 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.384112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xh4g8\""
Jul 28 23:23:10.392213 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.392188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmcnl"
Jul 28 23:23:10.469029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.468959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845" event={"ID":"ec64a69f-f83c-4799-9452-6afc6af4e254","Type":"ContainerStarted","Data":"912528c3728cc81c49b0df0b8e52d024bd722533aa9eeb9989bd2a91cca38ebf"}
Jul 28 23:23:10.536516 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:10.536485    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmcnl"]
Jul 28 23:23:10.539713 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:10.539688    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a684e9_2655_488b_bfc4_f8a7d89359f6.slice/crio-4d1d17dfd54082adf23cb35eaef867a068444f1e850ae23882c39f11e92010a5 WatchSource:0}: Error finding container 4d1d17dfd54082adf23cb35eaef867a068444f1e850ae23882c39f11e92010a5: Status 404 returned error can't find the container with id 4d1d17dfd54082adf23cb35eaef867a068444f1e850ae23882c39f11e92010a5
Jul 28 23:23:11.347167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:11.347131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:11.448119 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:11.448083    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:11.482148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:11.481531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmcnl" event={"ID":"d9a684e9-2655-488b-bfc4-f8a7d89359f6","Type":"ContainerStarted","Data":"4d1d17dfd54082adf23cb35eaef867a068444f1e850ae23882c39f11e92010a5"}
Jul 28 23:23:12.486329 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:12.486288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845" event={"ID":"ec64a69f-f83c-4799-9452-6afc6af4e254","Type":"ContainerStarted","Data":"4b92feabf31394aa643af6713e96a75d69323869834fa28d3430d4965c7f89cd"}
Jul 28 23:23:12.486792 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:12.486494    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:12.492006 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:12.491980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845"
Jul 28 23:23:12.500976 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:12.500922    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xs845" podStartSLOduration=2.131786409 podStartE2EDuration="3.500904034s" totalImagesPullingTime="1.369117625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:12.499392353 +0000 UTC m=+93.026441613" watchObservedRunningTime="2026-07-28 23:23:12.500904034 +0000 UTC m=+93.027953298"
Jul 28 23:23:13.320712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.320676    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:13.358807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.358776    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"]
Jul 28 23:23:13.374778 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.374746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"]
Jul 28 23:23:13.374930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.374844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.377314 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.377288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 23:23:13.377481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.377289    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 23:23:13.377481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.377425    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 23:23:13.378533 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.378508    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 23:23:13.378632 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.378534    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 23:23:13.378632 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.378550    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s2qpw\""
Jul 28 23:23:13.392970 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.392944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:13.461660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.461626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.461816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.461692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht7jk\" (UniqueName: \"kubernetes.io/projected/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-kube-api-access-ht7jk\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.461816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.461741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.461912 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.461811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.470996 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.470960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:13.562499 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.562451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht7jk\" (UniqueName: \"kubernetes.io/projected/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-kube-api-access-ht7jk\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.562952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.562527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.562952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.562559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.562952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.562657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.563483 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.563456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.565374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.565334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.565573 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.565555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.570230 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.570207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht7jk\" (UniqueName: \"kubernetes.io/projected/aef6b6fa-d0ed-4dff-869d-9b8dec920a2a-kube-api-access-ht7jk\") pod \"prometheus-operator-84d69f794d-sjw6s\" (UID: \"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:13.572320 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.572240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:13.685093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:13.685060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"
Jul 28 23:23:15.530593 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:15.530567    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sjw6s"]
Jul 28 23:23:15.534256 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:15.534232    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef6b6fa_d0ed_4dff_869d_9b8dec920a2a.slice/crio-c2fb22f86ec9a362be68092ae617417b53e27fc33f1c23c0cc4eacaa1c4d62bb WatchSource:0}: Error finding container c2fb22f86ec9a362be68092ae617417b53e27fc33f1c23c0cc4eacaa1c4d62bb: Status 404 returned error can't find the container with id c2fb22f86ec9a362be68092ae617417b53e27fc33f1c23c0cc4eacaa1c4d62bb
Jul 28 23:23:15.951138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:15.951104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:16.000092 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:16.000060    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:16.504056 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:16.504017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmcnl" event={"ID":"d9a684e9-2655-488b-bfc4-f8a7d89359f6","Type":"ContainerStarted","Data":"26aa685b6bea75abedded2eb4938cc9bb3e9df58eebb55e3f81932d3452f05fe"}
Jul 28 23:23:16.505889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:16.505817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s" event={"ID":"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a","Type":"ContainerStarted","Data":"c2fb22f86ec9a362be68092ae617417b53e27fc33f1c23c0cc4eacaa1c4d62bb"}
Jul 28 23:23:18.203117 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.203076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:18.205768 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.205740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"image-registry-ddc78c9cb-6mgmp\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") " pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:18.304446 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.304361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:23:18.306576 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.306552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4cd0759-0e50-4f91-82b7-2d12fc26c4c9-metrics-tls\") pod \"dns-default-6rt8g\" (UID: \"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9\") " pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:23:18.492211 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.492178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gxdzj\""
Jul 28 23:23:18.500306 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.500278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:18.508502 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.508476    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7kmkd\""
Jul 28 23:23:18.516579 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:18.516554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:23:19.197725 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.197688    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6rt8g"]
Jul 28 23:23:19.200801 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:19.200771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cd0759_0e50_4f91_82b7_2d12fc26c4c9.slice/crio-c14a3d8d08e0a443074f78f649a85340cca54c821468eb3e67f6d95eb4c8c2f8 WatchSource:0}: Error finding container c14a3d8d08e0a443074f78f649a85340cca54c821468eb3e67f6d95eb4c8c2f8: Status 404 returned error can't find the container with id c14a3d8d08e0a443074f78f649a85340cca54c821468eb3e67f6d95eb4c8c2f8
Jul 28 23:23:19.219282 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.219216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:23:19.223267 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:19.223235    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfddade49_3504_4bdd_9164_cfd866172a4c.slice/crio-6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f WatchSource:0}: Error finding container 6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f: Status 404 returned error can't find the container with id 6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f
Jul 28 23:23:19.299795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.299608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:19.405810 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.405778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:19.518162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.518127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s" event={"ID":"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a","Type":"ContainerStarted","Data":"b6e48ebfe363632c88b7eeea9fad233d5f43cd5e46ac0caf9d6a41b925c1796d"}
Jul 28 23:23:19.518361 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.518195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s" event={"ID":"aef6b6fa-d0ed-4dff-869d-9b8dec920a2a","Type":"ContainerStarted","Data":"fcda98001adc7ea15a19e35c2c26305d0e843870fc97a59f290a654a449cde4d"}
Jul 28 23:23:19.519364 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.519314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6rt8g" event={"ID":"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9","Type":"ContainerStarted","Data":"c14a3d8d08e0a443074f78f649a85340cca54c821468eb3e67f6d95eb4c8c2f8"}
Jul 28 23:23:19.520868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.520838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" event={"ID":"fddade49-3504-4bdd-9164-cfd866172a4c","Type":"ContainerStarted","Data":"5ae330fe08b471bc3ae965b818819932a2030a96dacbcf2a0a13b737a31cb383"}
Jul 28 23:23:19.520959 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.520885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" event={"ID":"fddade49-3504-4bdd-9164-cfd866172a4c","Type":"ContainerStarted","Data":"6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f"}
Jul 28 23:23:19.520959 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.520916    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:19.522802 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.522770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmcnl" event={"ID":"d9a684e9-2655-488b-bfc4-f8a7d89359f6","Type":"ContainerStarted","Data":"6cf49b289372d92f4f75c3cef97b09e7628652e5573869fd86edcb554247a99c"}
Jul 28 23:23:19.540205 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.539792    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-sjw6s" podStartSLOduration=3.037736035 podStartE2EDuration="6.539771994s" totalImagesPullingTime="3.502035959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:19.539657364 +0000 UTC m=+100.066706628" watchObservedRunningTime="2026-07-28 23:23:19.539771994 +0000 UTC m=+100.066821271"
Jul 28 23:23:19.559559 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:19.559463    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" podStartSLOduration=99.5594493 podStartE2EDuration="1m39.5594493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:19.55894864 +0000 UTC m=+100.085997905" watchObservedRunningTime="2026-07-28 23:23:19.5594493 +0000 UTC m=+100.086498565"
Jul 28 23:23:20.112136 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:20.112110    2564 scope.go:117] "RemoveContainer" containerID="dd33d6c03f7313b2f137b5e467cc005f9cf87cb90dc5926904ae576f85ccc531"
Jul 28 23:23:20.528798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:20.528719    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:23:20.529201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:20.528914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" event={"ID":"fef84116-f628-4535-a269-2e5bebfe6200","Type":"ContainerStarted","Data":"1ef11390296534bb4f5c5d344831044b12966fc850d01158dbb296046f200658"}
Jul 28 23:23:20.529741 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:20.529641    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:23:20.546359 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:20.546285    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg" podStartSLOduration=44.18147885 podStartE2EDuration="46.546264529s" totalImagesPullingTime="2.364785679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:20.544818342 +0000 UTC m=+101.071867607" watchObservedRunningTime="2026-07-28 23:23:20.546264529 +0000 UTC m=+101.073313795"
Jul 28 23:23:21.467602 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.467566    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:21.526116 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.526084    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-l26sg"
Jul 28 23:23:21.719949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.719675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:21.724059 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.724034    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-jhdmx"]
Jul 28 23:23:21.734759 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.734734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:21.737416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.737308    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 23:23:21.737580 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.737535    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 23:23:21.737658 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.737409    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-h2phq\""
Jul 28 23:23:21.738042 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.738010    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jhdmx"]
Jul 28 23:23:21.774787 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.774748    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"]
Jul 28 23:23:21.780065 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.780038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.783071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.782880    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 23:23:21.783383 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.783362    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 23:23:21.783503 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.783363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zlgbz\""
Jul 28 23:23:21.788582 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.788521    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"]
Jul 28 23:23:21.798587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.798560    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"]
Jul 28 23:23:21.803908 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.803882    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.806291 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.806273    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 23:23:21.806598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.806578    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 23:23:21.806701 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.806623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-g2rb5\""
Jul 28 23:23:21.806701 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.806675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 23:23:21.815599 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.815411    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w84mq"]
Jul 28 23:23:21.820936 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.820907    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"]
Jul 28 23:23:21.821049 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.821036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.823940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.823655    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 23:23:21.823940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.823665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 23:23:21.823940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.823717    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 23:23:21.823940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.823739    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8ncnk\""
Jul 28 23:23:21.824580 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.824558    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:21.835577 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.835699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.835699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdrfn\" (UniqueName: \"kubernetes.io/projected/7e62e3d3-a42f-486e-bb31-6ecb6132e700-kube-api-access-tdrfn\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.835817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79mq6\" (UniqueName: \"kubernetes.io/projected/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-api-access-79mq6\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.835817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.835817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.835817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flq4n\" (UniqueName: \"kubernetes.io/projected/12a4e6aa-0230-4af6-9b71-b2a34df9d6a7-kube-api-access-flq4n\") pod \"downloads-7c97dc4555-jhdmx\" (UID: \"12a4e6aa-0230-4af6-9b71-b2a34df9d6a7\") " pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:21.836024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c8d1e37-645f-43c8-a575-edc9bb4fa514-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.836024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.836024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e62e3d3-a42f-486e-bb31-6ecb6132e700-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.836024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.835983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.891628 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.891561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:21.937414 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c8d1e37-645f-43c8-a575-edc9bb4fa514-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.937598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.937598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.937598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e62e3d3-a42f-486e-bb31-6ecb6132e700-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.937598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.937972 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.937946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c8d1e37-645f-43c8-a575-edc9bb4fa514-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938057 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-root\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938103 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5tf\" (UniqueName: \"kubernetes.io/projected/3502deed-e2b6-4b35-a431-b774b5c5e9ae-kube-api-access-cn5tf\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938152 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938152 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-metrics-client-ca\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdrfn\" (UniqueName: \"kubernetes.io/projected/7e62e3d3-a42f-486e-bb31-6ecb6132e700-kube-api-access-tdrfn\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.938260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-sys\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79mq6\" (UniqueName: \"kubernetes.io/projected/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-api-access-79mq6\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-textfile\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flq4n\" (UniqueName: \"kubernetes.io/projected/12a4e6aa-0230-4af6-9b71-b2a34df9d6a7-kube-api-access-flq4n\") pod \"downloads-7c97dc4555-jhdmx\" (UID: \"12a4e6aa-0230-4af6-9b71-b2a34df9d6a7\") " pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-wtmp\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938600    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:21.938678    2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:21.938739    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls podName:4c8d1e37-645f-43c8-a575-edc9bb4fa514 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:22.438722438 +0000 UTC m=+102.965771684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-ghshp" (UID: "4c8d1e37-645f-43c8-a575-edc9bb4fa514") : secret "kube-state-metrics-tls" not found
Jul 28 23:23:21.938816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.938753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e62e3d3-a42f-486e-bb31-6ecb6132e700-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.939124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.939006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.940725 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.940691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.941112 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.941090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e62e3d3-a42f-486e-bb31-6ecb6132e700-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.941263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.941246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.950590 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.950545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79mq6\" (UniqueName: \"kubernetes.io/projected/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-api-access-79mq6\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:21.951915 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.951863    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdrfn\" (UniqueName: \"kubernetes.io/projected/7e62e3d3-a42f-486e-bb31-6ecb6132e700-kube-api-access-tdrfn\") pod \"openshift-state-metrics-b8f49f5c6-mkjdj\" (UID: \"7e62e3d3-a42f-486e-bb31-6ecb6132e700\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:21.952014 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.951965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flq4n\" (UniqueName: \"kubernetes.io/projected/12a4e6aa-0230-4af6-9b71-b2a34df9d6a7-kube-api-access-flq4n\") pod \"downloads-7c97dc4555-jhdmx\" (UID: \"12a4e6aa-0230-4af6-9b71-b2a34df9d6a7\") " pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:21.986666 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:21.986578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.039881 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.039843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-root\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.039881 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.039890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5tf\" (UniqueName: \"kubernetes.io/projected/3502deed-e2b6-4b35-a431-b774b5c5e9ae-kube-api-access-cn5tf\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.039930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-metrics-client-ca\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.039965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.039994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-sys\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.040031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-textfile\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.040059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-wtmp\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.040322 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.040082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.040526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-root\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.040128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:22.040999    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.041012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-sys\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.041038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-accelerators-collector-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:22.041064    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls podName:3502deed-e2b6-4b35-a431-b774b5c5e9ae nodeName:}" failed. No retries permitted until 2026-07-28 23:23:22.541046893 +0000 UTC m=+103.068096157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls") pod "node-exporter-w84mq" (UID: "3502deed-e2b6-4b35-a431-b774b5c5e9ae") : secret "node-exporter-tls" not found
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.041696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3502deed-e2b6-4b35-a431-b774b5c5e9ae-metrics-client-ca\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.042564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-wtmp\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.042668 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.042633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-textfile\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.043947 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.043902    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.046764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.046739    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:22.052169 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.052143    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5tf\" (UniqueName: \"kubernetes.io/projected/3502deed-e2b6-4b35-a431-b774b5c5e9ae-kube-api-access-cn5tf\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.086214 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.086177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.092031 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.092004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"
Jul 28 23:23:22.186405 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.186366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.308886 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.308844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.377194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.377161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.402519 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.402490    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-djhbc"
Jul 28 23:23:22.445289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.445216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:22.446492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.446454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.448335 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.448304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c8d1e37-645f-43c8-a575-edc9bb4fa514-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ghshp\" (UID: \"4c8d1e37-645f-43c8-a575-edc9bb4fa514\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:22.545877 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.545847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.548046 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.548018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.549572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.549546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3502deed-e2b6-4b35-a431-b774b5c5e9ae-node-exporter-tls\") pod \"node-exporter-w84mq\" (UID: \"3502deed-e2b6-4b35-a431-b774b5c5e9ae\") " pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.682388 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.682290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.716578 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.716543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"
Jul 28 23:23:22.732406 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.732377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w84mq"
Jul 28 23:23:22.782454 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.782417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.823003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.822969    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:23:22.856712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.856679    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:23:22.857519 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.856793    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.858989 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.858960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 23:23:22.859392 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 23:23:22.859392 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wzx68\""
Jul 28 23:23:22.859628 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859614    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 23:23:22.859692 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859641    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 23:23:22.859748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859641    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 23:23:22.859797 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859759    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 23:23:22.859797 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859782    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 23:23:22.859929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859834    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 23:23:22.859929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.859901    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 23:23:22.882481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.882456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:22.949785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6k8\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949993 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949867    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949993 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949993 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.949993 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.949978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950250 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:22.950250 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:22.950177    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051099 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051099 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051462 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051462 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051572    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051711 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:23:23.051684    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle podName:96c61502-f622-4eab-928c-1f5c1bc09ec6 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:23.551662877 +0000 UTC m=+104.078712133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6") : configmap references non-existent config key: ca-bundle.crt
Jul 28 23:23:23.051921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.051921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.051770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6k8\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.052037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.052011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.054399 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.054372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.054746 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.054538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.054746 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.054717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.054915 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.054848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.055118 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.055095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.055189 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.055128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.055309 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.055293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.055410 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.055394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.056028 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.056008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.060293 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.060273    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6k8\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.083798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.083745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.281680 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.281607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.444451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.444419    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:23:23.491852 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:23.491788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3502deed_e2b6_4b35_a431_b774b5c5e9ae.slice/crio-7dce0824f610323144ffdbfc7ebfb3766bbd609e3252a1fbf648d84c77917277 WatchSource:0}: Error finding container 7dce0824f610323144ffdbfc7ebfb3766bbd609e3252a1fbf648d84c77917277: Status 404 returned error can't find the container with id 7dce0824f610323144ffdbfc7ebfb3766bbd609e3252a1fbf648d84c77917277
Jul 28 23:23:23.526537 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.526452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.564207 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.558895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.564207 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.560258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.587798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.587758    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w84mq" event={"ID":"3502deed-e2b6-4b35-a431-b774b5c5e9ae","Type":"ContainerStarted","Data":"7dce0824f610323144ffdbfc7ebfb3766bbd609e3252a1fbf648d84c77917277"}
Jul 28 23:23:23.626215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.626151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.660777 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.660709    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jhdmx"]
Jul 28 23:23:23.664820 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:23.664784    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a4e6aa_0230_4af6_9b71_b2a34df9d6a7.slice/crio-aded280afe7f3cbd0ed3c2e7d848823f6aefefedec7130a139b5f78a2810de57 WatchSource:0}: Error finding container aded280afe7f3cbd0ed3c2e7d848823f6aefefedec7130a139b5f78a2810de57: Status 404 returned error can't find the container with id aded280afe7f3cbd0ed3c2e7d848823f6aefefedec7130a139b5f78a2810de57
Jul 28 23:23:23.687846 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.682068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp"]
Jul 28 23:23:23.689638 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.689609    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.690602 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:23.690576    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8d1e37_645f_43c8_a575_edc9bb4fa514.slice/crio-67abcaa09aafd48bc13e1c459b3465ca9644c373af69bb54c954ea9b69fe3db6 WatchSource:0}: Error finding container 67abcaa09aafd48bc13e1c459b3465ca9644c373af69bb54c954ea9b69fe3db6: Status 404 returned error can't find the container with id 67abcaa09aafd48bc13e1c459b3465ca9644c373af69bb54c954ea9b69fe3db6
Jul 28 23:23:23.703755 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.703728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj"]
Jul 28 23:23:23.706891 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:23.706851    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e62e3d3_a42f_486e_bb31_6ecb6132e700.slice/crio-beb26145af724a9cb8e18c2ce2b901583ee09b096767d2c4c0735463c8422a05 WatchSource:0}: Error finding container beb26145af724a9cb8e18c2ce2b901583ee09b096767d2c4c0735463c8422a05: Status 404 returned error can't find the container with id beb26145af724a9cb8e18c2ce2b901583ee09b096767d2c4c0735463c8422a05
Jul 28 23:23:23.768715 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.768692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:23:23.830625 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.830558    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:23.934414 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:23.934381    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:23:23.936872 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:23.936832    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c61502_f622_4eab_928c_1f5c1bc09ec6.slice/crio-c4a657ac8068ef298cf6ea9b000c50523a142c62b54499ee19b01a051ab8a3ff WatchSource:0}: Error finding container c4a657ac8068ef298cf6ea9b000c50523a142c62b54499ee19b01a051ab8a3ff: Status 404 returned error can't find the container with id c4a657ac8068ef298cf6ea9b000c50523a142c62b54499ee19b01a051ab8a3ff
Jul 28 23:23:24.010245 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.010207    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:24.596998 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.596893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6rt8g" event={"ID":"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9","Type":"ContainerStarted","Data":"d8e99de2c2dc8d17d47dc96fefa9c0c536431de9647dfc8254ae85c58593ee72"}
Jul 28 23:23:24.596998 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.596963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6rt8g" event={"ID":"d4cd0759-0e50-4f91-82b7-2d12fc26c4c9","Type":"ContainerStarted","Data":"22f46085a16ebaf2cb267c758d9388cecb41987cf87ab9b2d38e32197b0322f9"}
Jul 28 23:23:24.597256 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.597058    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:23:24.599564 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.599538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj" event={"ID":"7e62e3d3-a42f-486e-bb31-6ecb6132e700","Type":"ContainerStarted","Data":"85fcb9a70c87c35e77fed89f38f38cbead28c6197f3c00b39aa8fbc26237f568"}
Jul 28 23:23:24.599698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.599595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj" event={"ID":"7e62e3d3-a42f-486e-bb31-6ecb6132e700","Type":"ContainerStarted","Data":"2aa0ca54fd5976a7197c90c4b4c28a04c161457ca9291e51f3425c515804f6ab"}
Jul 28 23:23:24.599698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.599623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj" event={"ID":"7e62e3d3-a42f-486e-bb31-6ecb6132e700","Type":"ContainerStarted","Data":"beb26145af724a9cb8e18c2ce2b901583ee09b096767d2c4c0735463c8422a05"}
Jul 28 23:23:24.600908 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.600868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp" event={"ID":"4c8d1e37-645f-43c8-a575-edc9bb4fa514","Type":"ContainerStarted","Data":"67abcaa09aafd48bc13e1c459b3465ca9644c373af69bb54c954ea9b69fe3db6"}
Jul 28 23:23:24.603432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.603204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmcnl" event={"ID":"d9a684e9-2655-488b-bfc4-f8a7d89359f6","Type":"ContainerStarted","Data":"777226ee1fdd0b869fb20274759461d801185fdf793d35d353f54adb90b31163"}
Jul 28 23:23:24.604618 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.604593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"c4a657ac8068ef298cf6ea9b000c50523a142c62b54499ee19b01a051ab8a3ff"}
Jul 28 23:23:24.605800 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.605775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jhdmx" event={"ID":"12a4e6aa-0230-4af6-9b71-b2a34df9d6a7","Type":"ContainerStarted","Data":"aded280afe7f3cbd0ed3c2e7d848823f6aefefedec7130a139b5f78a2810de57"}
Jul 28 23:23:24.613670 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.613628    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6rt8g" podStartSLOduration=66.327725848 podStartE2EDuration="1m10.613613207s" totalImagesPullingTime="4.285887359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:24.612461729 +0000 UTC m=+105.139510993" watchObservedRunningTime="2026-07-28 23:23:24.613613207 +0000 UTC m=+105.140662473"
Jul 28 23:23:24.631724 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.631540    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nmcnl" podStartSLOduration=33.826278072 podStartE2EDuration="46.631519067s" totalImagesPullingTime="12.805240995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:24.629791844 +0000 UTC m=+105.156841109" watchObservedRunningTime="2026-07-28 23:23:24.631519067 +0000 UTC m=+105.158568330"
Jul 28 23:23:24.673726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.673696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:24.774482 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:24.774446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:25.026986 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.026917    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:25.128610 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.128473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:25.227011 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.226897    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:25.338767 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.338675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:25.612830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.612791    2564 generic.go:356] "Generic (PLEG): container finished" podID="3502deed-e2b6-4b35-a431-b774b5c5e9ae" containerID="37ea0d113e4bfef6aa24788477333cae4127b81dbe594262e55546a85fb231be" exitCode=0
Jul 28 23:23:25.613038 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:25.613007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w84mq" event={"ID":"3502deed-e2b6-4b35-a431-b774b5c5e9ae","Type":"ContainerDied","Data":"37ea0d113e4bfef6aa24788477333cae4127b81dbe594262e55546a85fb231be"}
Jul 28 23:23:26.175672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.175637    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86598459fb-rfn8p"]
Jul 28 23:23:26.195163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.195116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86598459fb-rfn8p"]
Jul 28 23:23:26.195383 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.195235    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.197962 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.197940    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 23:23:26.199660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.199421    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 23:23:26.199660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.199444    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 23:23:26.199660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.199466    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kgt7r\""
Jul 28 23:23:26.199660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.199515    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 23:23:26.199660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.199580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c9aao54j5fqbd\""
Jul 28 23:23:26.289889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.289856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-audit-log\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.289913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-tls\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.290002    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l9j4\" (UniqueName: \"kubernetes.io/projected/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-kube-api-access-6l9j4\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.290042    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-metrics-server-audit-profiles\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290241 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.290147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-client-certs\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290241 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.290175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.290241 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.290206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-client-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.390951 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.390861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-client-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-audit-log\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-tls\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391251 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l9j4\" (UniqueName: \"kubernetes.io/projected/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-kube-api-access-6l9j4\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391251 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-metrics-server-audit-profiles\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391321 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-client-certs\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.391321 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.392010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.391943    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-audit-log\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.392010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.392000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.399145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.393448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-metrics-server-audit-profiles\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.399145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.394151    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-client-ca-bundle\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.400513 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.400489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-client-certs\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.401097 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.401070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-secret-metrics-server-tls\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.408451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.408408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l9j4\" (UniqueName: \"kubernetes.io/projected/dad8d7b8-3ea0-4a40-8022-4adbacaa7acc-kube-api-access-6l9j4\") pod \"metrics-server-86598459fb-rfn8p\" (UID: \"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc\") " pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.494892 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.494789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:26.507941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.507799    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:26.549421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.549387    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:26.658054 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.657643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:26.798858 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:26.798833    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86598459fb-rfn8p"]
Jul 28 23:23:26.830620 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:26.830573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad8d7b8_3ea0_4a40_8022_4adbacaa7acc.slice/crio-fcad8871b9ce5491c758fa299614c784e83f66d06a0570240672dd13f9ec0594 WatchSource:0}: Error finding container fcad8871b9ce5491c758fa299614c784e83f66d06a0570240672dd13f9ec0594: Status 404 returned error can't find the container with id fcad8871b9ce5491c758fa299614c784e83f66d06a0570240672dd13f9ec0594
Jul 28 23:23:27.625429 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.625037    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w84mq" event={"ID":"3502deed-e2b6-4b35-a431-b774b5c5e9ae","Type":"ContainerStarted","Data":"25ce105bd02aea03a63b29802d29a862d0cf1a20eed58750e769fa17368a11b4"}
Jul 28 23:23:27.625429 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.625100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w84mq" event={"ID":"3502deed-e2b6-4b35-a431-b774b5c5e9ae","Type":"ContainerStarted","Data":"f096f243d5209f3538b76c9c643862f8d956591240285566c66d1504c2012339"}
Jul 28 23:23:27.628834 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.628807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj" event={"ID":"7e62e3d3-a42f-486e-bb31-6ecb6132e700","Type":"ContainerStarted","Data":"eeefae6f652e00c02b35f2bde95a6b8fd0541b45bbe53f3c5cc09f5f98b1d146"}
Jul 28 23:23:27.631898 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.631867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp" event={"ID":"4c8d1e37-645f-43c8-a575-edc9bb4fa514","Type":"ContainerStarted","Data":"dbd8ad30bbdedd6cb0ad3e89e91a8e51585bfbdd295b1cccd8cc0723a4a313ee"}
Jul 28 23:23:27.632023 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.631930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp" event={"ID":"4c8d1e37-645f-43c8-a575-edc9bb4fa514","Type":"ContainerStarted","Data":"72f06bf7ff518c0517118286287918251281a5a5fe0621102a98483d031bbaf5"}
Jul 28 23:23:27.632023 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.631968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp" event={"ID":"4c8d1e37-645f-43c8-a575-edc9bb4fa514","Type":"ContainerStarted","Data":"3700f3a79d0aa8c4673cb69bca1799883dc313d7db61eba83f0789d7548b0861"}
Jul 28 23:23:27.633587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.633551    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p" event={"ID":"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc","Type":"ContainerStarted","Data":"fcad8871b9ce5491c758fa299614c784e83f66d06a0570240672dd13f9ec0594"}
Jul 28 23:23:27.635561 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.635536    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8" exitCode=0
Jul 28 23:23:27.635692 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.635598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"}
Jul 28 23:23:27.645594 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.645366    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w84mq" podStartSLOduration=5.146284066 podStartE2EDuration="6.645322578s" totalImagesPullingTime="1.365821018s" totalInitContainerRuntime="133.217494ms" isStatefulPod=true podCreationTimestamp="2026-07-28 23:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:27.643490596 +0000 UTC m=+108.170539850" watchObservedRunningTime="2026-07-28 23:23:27.645322578 +0000 UTC m=+108.172371842"
Jul 28 23:23:27.662263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.662207    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ghshp" podStartSLOduration=3.719031846 podStartE2EDuration="6.662185246s" totalImagesPullingTime="2.9431534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:27.660792369 +0000 UTC m=+108.187841633" watchObservedRunningTime="2026-07-28 23:23:27.662185246 +0000 UTC m=+108.189234511"
Jul 28 23:23:27.702531 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.702467    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-mkjdj" podStartSLOduration=4.030154068 podStartE2EDuration="6.702449511s" totalImagesPullingTime="2.672295443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:27.700823136 +0000 UTC m=+108.227872402" watchObservedRunningTime="2026-07-28 23:23:27.702449511 +0000 UTC m=+108.229498752"
Jul 28 23:23:27.965774 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:27.965686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:29.652107 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.652071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p" event={"ID":"dad8d7b8-3ea0-4a40-8022-4adbacaa7acc","Type":"ContainerStarted","Data":"8b20183bdfa364aea0504882b853c02a7e6c5193d1c1605ed24529acdec8a1c0"}
Jul 28 23:23:29.657090 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.657060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"}
Jul 28 23:23:29.657185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.657139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"}
Jul 28 23:23:29.657185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.657178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"}
Jul 28 23:23:29.657293 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.657212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"}
Jul 28 23:23:29.668704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:29.668636    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p" podStartSLOduration=1.7225232259999999 podStartE2EDuration="3.668615427s" totalImagesPullingTime="1.946092201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:29.668245981 +0000 UTC m=+110.195295244" watchObservedRunningTime="2026-07-28 23:23:29.668615427 +0000 UTC m=+110.195664691"
Jul 28 23:23:30.669739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:30.669651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"}
Jul 28 23:23:30.670173 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:30.669741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerStarted","Data":"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"}
Jul 28 23:23:30.701250 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:30.701195    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.637903661 podStartE2EDuration="8.701176663s" totalImagesPullingTime="5.029223646s" totalInitContainerRuntime="34.049356ms" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:30.699979817 +0000 UTC m=+111.227029080" watchObservedRunningTime="2026-07-28 23:23:30.701176663 +0000 UTC m=+111.228225927"
Jul 28 23:23:34.615608 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:34.615569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:34.615608 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:34.615592    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6rt8g"
Jul 28 23:23:35.742032 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:35.741998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:35.843032 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:35.842984    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:36.582674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.582639    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:23:36.587405 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.587382    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.591310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.591283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8sgs4\""
Jul 28 23:23:36.591459 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.591375    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 23:23:36.591542 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.591459    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 23:23:36.591542 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.591484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 23:23:36.591652 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.591570    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 23:23:36.592040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.592008    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 23:23:36.596275 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.596092    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 23:23:36.596779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.596761    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:23:36.695024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.694987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m84pp\" (UniqueName: \"kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.695503 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.695248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.796628 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796640    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m84pp\" (UniqueName: \"kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.796870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797541 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.797484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797541 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.797533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.797716 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.797635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.798040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.798019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.799694 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.799669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.799789 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.799766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.804832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.804810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m84pp\" (UniqueName: \"kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp\") pod \"console-68bb87f5d8-v952w\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") " pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:36.900319 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:36.900230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:37.810538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:37.810495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:37.865833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:37.865794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:38.108757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:38.108675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:38.178407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:38.178368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:41.018120 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:41.018078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:42.084940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.084914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:23:42.087287 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:42.087259    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c5ea1d_9249_408c_aa6b_600a604cdc29.slice/crio-94c5c3a8927d5dc36aaf655a311b1714b2f77af2b8823638a2939a68089f00a0 WatchSource:0}: Error finding container 94c5c3a8927d5dc36aaf655a311b1714b2f77af2b8823638a2939a68089f00a0: Status 404 returned error can't find the container with id 94c5c3a8927d5dc36aaf655a311b1714b2f77af2b8823638a2939a68089f00a0
Jul 28 23:23:42.215906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.215823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:42.728763 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.728729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jhdmx" event={"ID":"12a4e6aa-0230-4af6-9b71-b2a34df9d6a7","Type":"ContainerStarted","Data":"81167eb665c43c59cc314cdb31fa18d2e09e857d25c17c61c7e4cebe6e353e23"}
Jul 28 23:23:42.729313 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.729289    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:42.731014 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.730985    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bb87f5d8-v952w" event={"ID":"93c5ea1d-9249-408c-aa6b-600a604cdc29","Type":"ContainerStarted","Data":"94c5c3a8927d5dc36aaf655a311b1714b2f77af2b8823638a2939a68089f00a0"}
Jul 28 23:23:42.743878 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.743835    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-jhdmx"
Jul 28 23:23:42.747506 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:42.747273    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-jhdmx" podStartSLOduration=3.028779739 podStartE2EDuration="21.747253704s" totalImagesPullingTime="18.718473965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:42.745196899 +0000 UTC m=+123.272246175" watchObservedRunningTime="2026-07-28 23:23:42.747253704 +0000 UTC m=+123.274302968"
Jul 28 23:23:43.454122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:43.452705    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:43.723876 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:43.723711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:43.773810 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:43.773723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:46.508554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.508510    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:46.509015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.508606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:23:46.622235 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.622188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:46.672930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.672891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:46.752882 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.752844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bb87f5d8-v952w" event={"ID":"93c5ea1d-9249-408c-aa6b-600a604cdc29","Type":"ContainerStarted","Data":"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"}
Jul 28 23:23:46.770232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.770126    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68bb87f5d8-v952w" podStartSLOduration=6.962793229 podStartE2EDuration="10.770109059s" totalImagesPullingTime="3.80731583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:46.769096985 +0000 UTC m=+127.296146250" watchObservedRunningTime="2026-07-28 23:23:46.770109059 +0000 UTC m=+127.297158321"
Jul 28 23:23:46.901318 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.901277    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:46.901525 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.901383    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:46.907077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:46.907047    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:47.761314 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:47.761281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:23:48.464584 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.464541    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" podUID="fddade49-3504-4bdd-9164-cfd866172a4c" containerName="registry" containerID="cri-o://5ae330fe08b471bc3ae965b818819932a2030a96dacbcf2a0a13b737a31cb383" gracePeriod=30
Jul 28 23:23:48.763933 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.763896    2564 generic.go:356] "Generic (PLEG): container finished" podID="fddade49-3504-4bdd-9164-cfd866172a4c" containerID="5ae330fe08b471bc3ae965b818819932a2030a96dacbcf2a0a13b737a31cb383" exitCode=0
Jul 28 23:23:48.764380 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.763974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" event={"ID":"fddade49-3504-4bdd-9164-cfd866172a4c","Type":"ContainerDied","Data":"5ae330fe08b471bc3ae965b818819932a2030a96dacbcf2a0a13b737a31cb383"}
Jul 28 23:23:48.764380 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.764060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp" event={"ID":"fddade49-3504-4bdd-9164-cfd866172a4c","Type":"ContainerDied","Data":"6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f"}
Jul 28 23:23:48.764380 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.764131    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6edf7e57818182cb5308efd0d80e5d216488ed8acae10c511774076f20de473f"
Jul 28 23:23:48.764634 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.764614    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:48.809929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.809894    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.809942    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.809977    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810297 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.810265    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.810333    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.810439    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.810472    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.810577 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.810564    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sddg6\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6\") pod \"fddade49-3504-4bdd-9164-cfd866172a4c\" (UID: \"fddade49-3504-4bdd-9164-cfd866172a4c\") "
Jul 28 23:23:48.811059 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.811026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:23:48.812883 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.812853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:23:48.814946 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.814918    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:23:48.815536 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.815479    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:23:48.815853 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.815825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6" (OuterVolumeSpecName: "kube-api-access-sddg6") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "kube-api-access-sddg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:23:48.816922 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.816897    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:23:48.818172 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.818146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:23:48.823232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.823202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fddade49-3504-4bdd-9164-cfd866172a4c" (UID: "fddade49-3504-4bdd-9164-cfd866172a4c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:23:48.845444 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.845355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:48.911660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911621    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-registry-certificates\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911658    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-registry-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911675    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sddg6\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-kube-api-access-sddg6\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911685    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fddade49-3504-4bdd-9164-cfd866172a4c-bound-sa-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911695    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fddade49-3504-4bdd-9164-cfd866172a4c-ca-trust-extracted\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911708    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-image-registry-private-configuration\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911722    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fddade49-3504-4bdd-9164-cfd866172a4c-trusted-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:48.911901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:48.911735    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fddade49-3504-4bdd-9164-cfd866172a4c-installation-pull-secrets\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:23:49.768859 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:49.768822    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddc78c9cb-6mgmp"
Jul 28 23:23:49.791751 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:49.791723    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:23:49.795035 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:49.795009    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-ddc78c9cb-6mgmp"]
Jul 28 23:23:50.022298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.022204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:23:50.025004 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.024977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43cc8c79-993f-463f-9b5e-8aaaa4a9d068-metrics-certs\") pod \"network-metrics-daemon-bw89j\" (UID: \"43cc8c79-993f-463f-9b5e-8aaaa4a9d068\") " pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:23:50.115678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.115631    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fddade49-3504-4bdd-9164-cfd866172a4c" path="/var/lib/kubelet/pods/fddade49-3504-4bdd-9164-cfd866172a4c/volumes"
Jul 28 23:23:50.327381 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.327329    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plrk7\""
Jul 28 23:23:50.335512 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.335465    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bw89j"
Jul 28 23:23:50.538592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.538566    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bw89j"]
Jul 28 23:23:50.541394 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:23:50.541338    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cc8c79_993f_463f_9b5e_8aaaa4a9d068.slice/crio-980285538fe593585adf8024f382c052614ee2b0da0470183fed218c36b5b5f2 WatchSource:0}: Error finding container 980285538fe593585adf8024f382c052614ee2b0da0470183fed218c36b5b5f2: Status 404 returned error can't find the container with id 980285538fe593585adf8024f382c052614ee2b0da0470183fed218c36b5b5f2
Jul 28 23:23:50.742819 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.742734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:50.774254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.774212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bw89j" event={"ID":"43cc8c79-993f-463f-9b5e-8aaaa4a9d068","Type":"ContainerStarted","Data":"980285538fe593585adf8024f382c052614ee2b0da0470183fed218c36b5b5f2"}
Jul 28 23:23:50.791615 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:50.791584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:52.784507 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:52.784471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bw89j" event={"ID":"43cc8c79-993f-463f-9b5e-8aaaa4a9d068","Type":"ContainerStarted","Data":"a633597ffbbe176e1ab7f67fe9fa3a76864283c50782ef1278c6a9643b02e3b4"}
Jul 28 23:23:52.784879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:52.784554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bw89j" event={"ID":"43cc8c79-993f-463f-9b5e-8aaaa4a9d068","Type":"ContainerStarted","Data":"be9dbd03aca2da51e5b07a1d6b52b8b2779ee16bf6097e5606961fe86edfdb6c"}
Jul 28 23:23:52.801144 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:52.801096    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bw89j" podStartSLOduration=131.322264886 podStartE2EDuration="2m12.801081171s" totalImagesPullingTime="1.478816285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:52.799613596 +0000 UTC m=+133.326662859" watchObservedRunningTime="2026-07-28 23:23:52.801081171 +0000 UTC m=+133.328130434"
Jul 28 23:23:52.877753 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:52.877718    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:52.941746 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:52.941712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:57.395564 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:57.395526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:57.496710 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:57.496676    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:57.548809 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:57.548775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:57.780860 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:57.780783    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:57.868534 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:57.868502    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:23:58.808800 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:58.808763    2564 generic.go:356] "Generic (PLEG): container finished" podID="8f91cf43-36ec-47cb-870c-57d39b4b4de2" containerID="c048668ebca4490146a36467904439e34fca5cdd617a60425d9fc9ed24e174d2" exitCode=0
Jul 28 23:23:58.809219 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:58.808837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8gkcs" event={"ID":"8f91cf43-36ec-47cb-870c-57d39b4b4de2","Type":"ContainerDied","Data":"c048668ebca4490146a36467904439e34fca5cdd617a60425d9fc9ed24e174d2"}
Jul 28 23:23:58.809219 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:58.809189    2564 scope.go:117] "RemoveContainer" containerID="c048668ebca4490146a36467904439e34fca5cdd617a60425d9fc9ed24e174d2"
Jul 28 23:23:59.814441 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:23:59.814408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-8gkcs" event={"ID":"8f91cf43-36ec-47cb-870c-57d39b4b4de2","Type":"ContainerStarted","Data":"a596576b1d68a019d45e3122022a39860214f2203f62382357856d500822ba6a"}
Jul 28 23:24:05.743322 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:05.743286    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:05.795309 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:05.795273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:06.513708 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:06.513678    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:24:06.517788 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:06.517764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86598459fb-rfn8p"
Jul 28 23:24:29.970513 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:29.970474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:37.820398 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:37.820311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:37.872944 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:37.872913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:38.361987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.361942    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:24:38.362488 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.362456    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fddade49-3504-4bdd-9164-cfd866172a4c" containerName="registry"
Jul 28 23:24:38.362488 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.362482    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fddade49-3504-4bdd-9164-cfd866172a4c" containerName="registry"
Jul 28 23:24:38.362757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.362583    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fddade49-3504-4bdd-9164-cfd866172a4c" containerName="registry"
Jul 28 23:24:38.365327 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.365306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.376629 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.376589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:24:38.406912 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.406877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:38.445244 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445355    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445384    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.445427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.445411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx5kl\" (UniqueName: \"kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.465936 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.465900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:38.546069 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546078    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx5kl\" (UniqueName: \"kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546515 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546515 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546894 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.546989 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.546926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.547073 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.547056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.547137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.547108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.548666 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.548644    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.548773 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.548756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.553588 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.553567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx5kl\" (UniqueName: \"kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl\") pod \"console-6c7d57769f-v5nj4\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") " pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.677971 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.677876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:38.811931 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.811902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:24:38.814285 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:24:38.814258    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod862b97c6_82c5_453e_9272_b59e4d7f389a.slice/crio-a925ececb70289d6ff917e56718df4c55f1aa825e788ad19e5e10556d617b138 WatchSource:0}: Error finding container a925ececb70289d6ff917e56718df4c55f1aa825e788ad19e5e10556d617b138: Status 404 returned error can't find the container with id a925ececb70289d6ff917e56718df4c55f1aa825e788ad19e5e10556d617b138
Jul 28 23:24:38.976186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.976084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d57769f-v5nj4" event={"ID":"862b97c6-82c5-453e-9272-b59e4d7f389a","Type":"ContainerStarted","Data":"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"}
Jul 28 23:24:38.976186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.976142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d57769f-v5nj4" event={"ID":"862b97c6-82c5-453e-9272-b59e4d7f389a","Type":"ContainerStarted","Data":"a925ececb70289d6ff917e56718df4c55f1aa825e788ad19e5e10556d617b138"}
Jul 28 23:24:38.992290 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:38.992239    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c7d57769f-v5nj4" podStartSLOduration=0.992222299 podStartE2EDuration="992.222299ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:24:38.991420565 +0000 UTC m=+179.518469829" watchObservedRunningTime="2026-07-28 23:24:38.992222299 +0000 UTC m=+179.519271562"
Jul 28 23:24:42.067879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.067842    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:42.068288 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068124    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="alertmanager" containerID="cri-o://05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009" gracePeriod=120
Jul 28 23:24:42.068288 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068200    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-web" containerID="cri-o://54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e" gracePeriod=120
Jul 28 23:24:42.068288 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068219    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy" containerID="cri-o://221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c" gracePeriod=120
Jul 28 23:24:42.068288 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068260    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="config-reloader" containerID="cri-o://01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24" gracePeriod=120
Jul 28 23:24:42.068288 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068279    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="prom-label-proxy" containerID="cri-o://1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542" gracePeriod=120
Jul 28 23:24:42.068591 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.068198    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-metric" containerID="cri-o://b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b" gracePeriod=120
Jul 28 23:24:42.996879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996844    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542" exitCode=0
Jul 28 23:24:42.996879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996870    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c" exitCode=0
Jul 28 23:24:42.996879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996879    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24" exitCode=0
Jul 28 23:24:42.996879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996890    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009" exitCode=0
Jul 28 23:24:42.997164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"}
Jul 28 23:24:42.997164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.996973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"}
Jul 28 23:24:42.997164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.997002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"}
Jul 28 23:24:42.997164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:42.997024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"}
Jul 28 23:24:43.324203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.324176    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:43.390495 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390467    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390520    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390554    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390573    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390589    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390612    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.390661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.390649    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391059    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391124    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391189    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391185    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391229    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc6k8\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391255    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391530 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"96c61502-f622-4eab-928c-1f5c1bc09ec6\" (UID: \"96c61502-f622-4eab-928c-1f5c1bc09ec6\") "
Jul 28 23:24:43.391950 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.391616    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.392599 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.392570    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:24:43.393276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.392891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:24:43.393427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.393403    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:24:43.393913 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.393875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out" (OuterVolumeSpecName: "config-out") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:24:43.394178 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.394147    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume" (OuterVolumeSpecName: "config-volume") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.394593 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.394528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.394910 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.394884    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.395112 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.395092    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.395196 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.395096    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.395494 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.395476    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8" (OuterVolumeSpecName: "kube-api-access-rc6k8") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "kube-api-access-rc6k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:24:43.399527 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.399398    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.407319 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.407290    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config" (OuterVolumeSpecName: "web-config") pod "96c61502-f622-4eab-928c-1f5c1bc09ec6" (UID: "96c61502-f622-4eab-928c-1f5c1bc09ec6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:24:43.492712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492675    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492706    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c61502-f622-4eab-928c-1f5c1bc09ec6-metrics-client-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492716    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-out\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492724    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-web-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492733    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc6k8\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-kube-api-access-rc6k8\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492742    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96c61502-f622-4eab-928c-1f5c1bc09ec6-alertmanager-main-db\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492751    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492759    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-cluster-tls-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492767    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492775    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492783    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96c61502-f622-4eab-928c-1f5c1bc09ec6-secret-alertmanager-main-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:43.492930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:43.492791    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96c61502-f622-4eab-928c-1f5c1bc09ec6-tls-assets\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:24:44.003835 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.003796    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b" exitCode=0
Jul 28 23:24:44.003835 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.003829    2564 generic.go:356] "Generic (PLEG): container finished" podID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerID="54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e" exitCode=0
Jul 28 23:24:44.004036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.003886    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.004036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.003890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"}
Jul 28 23:24:44.004036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.003989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"}
Jul 28 23:24:44.004036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.004018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96c61502-f622-4eab-928c-1f5c1bc09ec6","Type":"ContainerDied","Data":"c4a657ac8068ef298cf6ea9b000c50523a142c62b54499ee19b01a051ab8a3ff"}
Jul 28 23:24:44.004160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.004057    2564 scope.go:117] "RemoveContainer" containerID="1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"
Jul 28 23:24:44.011915 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.011796    2564 scope.go:117] "RemoveContainer" containerID="b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"
Jul 28 23:24:44.019084 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.019067    2564 scope.go:117] "RemoveContainer" containerID="221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"
Jul 28 23:24:44.026820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.026799    2564 scope.go:117] "RemoveContainer" containerID="54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"
Jul 28 23:24:44.027079 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.027058    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:44.031440 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.031412    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:44.034179 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.034165    2564 scope.go:117] "RemoveContainer" containerID="01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"
Jul 28 23:24:44.041057 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.041040    2564 scope.go:117] "RemoveContainer" containerID="05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"
Jul 28 23:24:44.048844 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.048825    2564 scope.go:117] "RemoveContainer" containerID="4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"
Jul 28 23:24:44.056215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.056195    2564 scope.go:117] "RemoveContainer" containerID="1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"
Jul 28 23:24:44.056510 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.056485    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542\": container with ID starting with 1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542 not found: ID does not exist" containerID="1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"
Jul 28 23:24:44.056664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.056520    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"} err="failed to get container status \"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542\": rpc error: code = NotFound desc = could not find container \"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542\": container with ID starting with 1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542 not found: ID does not exist"
Jul 28 23:24:44.056664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.056559    2564 scope.go:117] "RemoveContainer" containerID="b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"
Jul 28 23:24:44.057013 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.056985    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b\": container with ID starting with b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b not found: ID does not exist" containerID="b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"
Jul 28 23:24:44.057765 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057022    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"} err="failed to get container status \"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b\": rpc error: code = NotFound desc = could not find container \"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b\": container with ID starting with b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b not found: ID does not exist"
Jul 28 23:24:44.057765 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057048    2564 scope.go:117] "RemoveContainer" containerID="221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"
Jul 28 23:24:44.057765 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.057565    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c\": container with ID starting with 221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c not found: ID does not exist" containerID="221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"
Jul 28 23:24:44.057765 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057595    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"} err="failed to get container status \"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c\": rpc error: code = NotFound desc = could not find container \"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c\": container with ID starting with 221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c not found: ID does not exist"
Jul 28 23:24:44.057765 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057619    2564 scope.go:117] "RemoveContainer" containerID="54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"
Jul 28 23:24:44.058004 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.057943    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e\": container with ID starting with 54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e not found: ID does not exist" containerID="54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"
Jul 28 23:24:44.058004 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057964    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"} err="failed to get container status \"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e\": rpc error: code = NotFound desc = could not find container \"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e\": container with ID starting with 54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e not found: ID does not exist"
Jul 28 23:24:44.058004 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.057983    2564 scope.go:117] "RemoveContainer" containerID="01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"
Jul 28 23:24:44.058312 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.058282    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24\": container with ID starting with 01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24 not found: ID does not exist" containerID="01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"
Jul 28 23:24:44.058427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.058317    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"} err="failed to get container status \"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24\": rpc error: code = NotFound desc = could not find container \"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24\": container with ID starting with 01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24 not found: ID does not exist"
Jul 28 23:24:44.058427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.058372    2564 scope.go:117] "RemoveContainer" containerID="05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"
Jul 28 23:24:44.058669 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.058650    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009\": container with ID starting with 05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009 not found: ID does not exist" containerID="05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"
Jul 28 23:24:44.058731 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.058688    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"} err="failed to get container status \"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009\": rpc error: code = NotFound desc = could not find container \"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009\": container with ID starting with 05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009 not found: ID does not exist"
Jul 28 23:24:44.058731 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.058705    2564 scope.go:117] "RemoveContainer" containerID="4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"
Jul 28 23:24:44.059034 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:24:44.059014    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8\": container with ID starting with 4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8 not found: ID does not exist" containerID="4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"
Jul 28 23:24:44.059132 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059040    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"} err="failed to get container status \"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8\": rpc error: code = NotFound desc = could not find container \"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8\": container with ID starting with 4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8 not found: ID does not exist"
Jul 28 23:24:44.059132 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059058    2564 scope.go:117] "RemoveContainer" containerID="1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"
Jul 28 23:24:44.059330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059294    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542"} err="failed to get container status \"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542\": rpc error: code = NotFound desc = could not find container \"1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542\": container with ID starting with 1f72bdf9aff29fe8bb7d60c5ba0e74cdbcd34547c5fe5b398f23056cc6357542 not found: ID does not exist"
Jul 28 23:24:44.059330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059323    2564 scope.go:117] "RemoveContainer" containerID="b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"
Jul 28 23:24:44.059494 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059307    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:44.059767 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059750    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="prom-label-proxy"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059770    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="prom-label-proxy"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059784    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="init-config-reloader"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059792    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="init-config-reloader"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059802    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-web"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059810    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-web"
Jul 28 23:24:44.059832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059828    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059836    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059849    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="alertmanager"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059856    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="alertmanager"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059870    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="config-reloader"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059879    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="config-reloader"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059891    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-metric"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059899    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-metric"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059961    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-web"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059971    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="alertmanager"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059982    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy-metric"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.059993    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="prom-label-proxy"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.060002    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="kube-rbac-proxy"
Jul 28 23:24:44.060029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.060012    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" containerName="config-reloader"
Jul 28 23:24:44.061317 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061295    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b"} err="failed to get container status \"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b\": rpc error: code = NotFound desc = could not find container \"b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b\": container with ID starting with b36b38ca97a48d549f0fbeba40edc1375ad9dd52b4da4b101ddacad4dff1b77b not found: ID does not exist"
Jul 28 23:24:44.061416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061318    2564 scope.go:117] "RemoveContainer" containerID="221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"
Jul 28 23:24:44.061676 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061655    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c"} err="failed to get container status \"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c\": rpc error: code = NotFound desc = could not find container \"221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c\": container with ID starting with 221cc08ba012e49889d643bc0d8c382b2ea1624f94699b267a94322a5786f43c not found: ID does not exist"
Jul 28 23:24:44.061751 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061676    2564 scope.go:117] "RemoveContainer" containerID="54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"
Jul 28 23:24:44.061913 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061890    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e"} err="failed to get container status \"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e\": rpc error: code = NotFound desc = could not find container \"54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e\": container with ID starting with 54467ae1a831db0cb07c4704ba409ffd9ecabbf5ff66ae0909f531dad5533d1e not found: ID does not exist"
Jul 28 23:24:44.061959 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.061916    2564 scope.go:117] "RemoveContainer" containerID="01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"
Jul 28 23:24:44.062119 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.062098    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24"} err="failed to get container status \"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24\": rpc error: code = NotFound desc = could not find container \"01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24\": container with ID starting with 01aec31944fc6d774fb9fab6f4a472fc03976d3a535dba748d29b7a7a4007d24 not found: ID does not exist"
Jul 28 23:24:44.062167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.062119    2564 scope.go:117] "RemoveContainer" containerID="05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"
Jul 28 23:24:44.062370 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.062328    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009"} err="failed to get container status \"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009\": rpc error: code = NotFound desc = could not find container \"05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009\": container with ID starting with 05684edf9ac4bdff13b0d6eebc2d5659a643535377e260b59c24d9c7ae886009 not found: ID does not exist"
Jul 28 23:24:44.062449 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.062371    2564 scope.go:117] "RemoveContainer" containerID="4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"
Jul 28 23:24:44.062613 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.062592    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8"} err="failed to get container status \"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8\": rpc error: code = NotFound desc = could not find container \"4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8\": container with ID starting with 4068576bf9c54d1ae77ce59c039217cd1b87b14fb86d34e0a97554a74a5659e8 not found: ID does not exist"
Jul 28 23:24:44.065187 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.065170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.067671 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.067643    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 23:24:44.067895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.067877    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 23:24:44.067895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.067891    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 23:24:44.068048 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.067952    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 23:24:44.068048 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.068019    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 23:24:44.068154 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.068142    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 23:24:44.068450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.068433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 23:24:44.068655 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.068641    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wzx68\""
Jul 28 23:24:44.068957 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.068937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 23:24:44.072957 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.072913    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 23:24:44.074972 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.074952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:44.097581 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-config-volume\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-web-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-config-out\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.097930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.098102 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.098102 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.097980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.098102 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.098013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.098102 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.098092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dndmw\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-kube-api-access-dndmw\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.114535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.114506    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c61502-f622-4eab-928c-1f5c1bc09ec6" path="/var/lib/kubelet/pods/96c61502-f622-4eab-928c-1f5c1bc09ec6/volumes"
Jul 28 23:24:44.198869 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-config-volume\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.198869 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-web-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-config-out\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.198968    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dndmw\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-kube-api-access-dndmw\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.199611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.199339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.200403 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.200374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.200565 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.200471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59286e52-1968-4b6d-a03b-1185417b4adc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202114 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202250 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202566 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59286e52-1968-4b6d-a03b-1185417b4adc-config-out\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-web-config\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202640    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202866 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.202994 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.202975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-config-volume\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.203094 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.203073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.204205 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.204190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59286e52-1968-4b6d-a03b-1185417b4adc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.207460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.207436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dndmw\" (UniqueName: \"kubernetes.io/projected/59286e52-1968-4b6d-a03b-1185417b4adc-kube-api-access-dndmw\") pod \"alertmanager-main-0\" (UID: \"59286e52-1968-4b6d-a03b-1185417b4adc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.375187 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.375147    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 23:24:44.517899 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:44.517869    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 23:24:44.519708 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:24:44.519677    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59286e52_1968_4b6d_a03b_1185417b4adc.slice/crio-f49e1785b55e0332d31a153c5edc78226a5ea73ddb47093fefda94935956134a WatchSource:0}: Error finding container f49e1785b55e0332d31a153c5edc78226a5ea73ddb47093fefda94935956134a: Status 404 returned error can't find the container with id f49e1785b55e0332d31a153c5edc78226a5ea73ddb47093fefda94935956134a
Jul 28 23:24:45.011109 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:45.011067    2564 generic.go:356] "Generic (PLEG): container finished" podID="59286e52-1968-4b6d-a03b-1185417b4adc" containerID="23d4cfa2d812e19d33ca80f03f53871f24791482e465c8bf662949aae7469278" exitCode=0
Jul 28 23:24:45.011284 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:45.011155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerDied","Data":"23d4cfa2d812e19d33ca80f03f53871f24791482e465c8bf662949aae7469278"}
Jul 28 23:24:45.011284 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:45.011225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"f49e1785b55e0332d31a153c5edc78226a5ea73ddb47093fefda94935956134a"}
Jul 28 23:24:46.019248 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"52bd16f493cf9a57311d4f8cff61c14de002406ddd97caf62e9ae556c4f28200"}
Jul 28 23:24:46.019667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"28762c12ae1f72de9c6ed8519ee01808a32ef9bc910629b9505d82f7586c48a2"}
Jul 28 23:24:46.019667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"b1f864c27d692aba40ef5db296cd15ae9a07447fa3b65d1e25bbcdd43bd2c135"}
Jul 28 23:24:46.019667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"e5fd71da9a7dc04d5e8323e5546110b5fd415fbaadecacc9616238133aaa9ff0"}
Jul 28 23:24:46.019667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"d6028499589582239bbb091d39a2d4f7e0528cd3266bc6fc4227ebf807d77bc0"}
Jul 28 23:24:46.019667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.019428    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59286e52-1968-4b6d-a03b-1185417b4adc","Type":"ContainerStarted","Data":"b37c0b22d0cfe6b7c6cd23f2906b2c402d709419221a3694b69a6197e686e107"}
Jul 28 23:24:46.045519 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:46.045467    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.035730377 podStartE2EDuration="2.045450801s" totalImagesPullingTime="0s" totalInitContainerRuntime="9.720424ms" isStatefulPod=false podCreationTimestamp="2026-07-28 23:24:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:24:46.042989447 +0000 UTC m=+186.570038709" watchObservedRunningTime="2026-07-28 23:24:46.045450801 +0000 UTC m=+186.572500063"
Jul 28 23:24:48.678476 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:48.678428    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:48.678860 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:48.678500    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:48.683044 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:48.683013    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:49.036306 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.036218    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:24:49.077437 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.077397    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:24:49.175731 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.175698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:49.278868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.278827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:49.330492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.330460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:49.644389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.644284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:24:49.696433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:24:49.696402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:25:05.745315 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:05.745283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:25:05.845444 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:05.845408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:25:14.099376 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.099303    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68bb87f5d8-v952w" podUID="93c5ea1d-9249-408c-aa6b-600a604cdc29" containerName="console" containerID="cri-o://f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752" gracePeriod=15
Jul 28 23:25:14.334689 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.334661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:25:14.452073 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.451995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452073 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452036    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452073 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452055    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m84pp\" (UniqueName: \"kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452149    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452177    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452247    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config\") pod \"93c5ea1d-9249-408c-aa6b-600a604cdc29\" (UID: \"93c5ea1d-9249-408c-aa6b-600a604cdc29\") "
Jul 28 23:25:14.452569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452540    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:25:14.452704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca" (OuterVolumeSpecName: "service-ca") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:25:14.452764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452742    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config" (OuterVolumeSpecName: "console-config") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:25:14.452764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.452752    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:25:14.454563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.454539    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:25:14.454721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.454705    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:25:14.454765 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.454732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp" (OuterVolumeSpecName: "kube-api-access-m84pp") pod "93c5ea1d-9249-408c-aa6b-600a604cdc29" (UID: "93c5ea1d-9249-408c-aa6b-600a604cdc29"). InnerVolumeSpecName "kube-api-access-m84pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:25:14.553914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553873    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-trusted-ca-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.553914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553906    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.553914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553918    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.554160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553929    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-oauth-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.554160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553942    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93c5ea1d-9249-408c-aa6b-600a604cdc29-oauth-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.554160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553954    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93c5ea1d-9249-408c-aa6b-600a604cdc29-console-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:14.554160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:14.553965    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m84pp\" (UniqueName: \"kubernetes.io/projected/93c5ea1d-9249-408c-aa6b-600a604cdc29-kube-api-access-m84pp\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:25:15.136856 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.136821    2564 generic.go:356] "Generic (PLEG): container finished" podID="93c5ea1d-9249-408c-aa6b-600a604cdc29" containerID="f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752" exitCode=0
Jul 28 23:25:15.137273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.136868    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68bb87f5d8-v952w"
Jul 28 23:25:15.137273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.136889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bb87f5d8-v952w" event={"ID":"93c5ea1d-9249-408c-aa6b-600a604cdc29","Type":"ContainerDied","Data":"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"}
Jul 28 23:25:15.137273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.136930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68bb87f5d8-v952w" event={"ID":"93c5ea1d-9249-408c-aa6b-600a604cdc29","Type":"ContainerDied","Data":"94c5c3a8927d5dc36aaf655a311b1714b2f77af2b8823638a2939a68089f00a0"}
Jul 28 23:25:15.137273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.136952    2564 scope.go:117] "RemoveContainer" containerID="f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"
Jul 28 23:25:15.148135 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.148117    2564 scope.go:117] "RemoveContainer" containerID="f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"
Jul 28 23:25:15.148397 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:25:15.148378    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752\": container with ID starting with f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752 not found: ID does not exist" containerID="f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"
Jul 28 23:25:15.148457 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.148408    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752"} err="failed to get container status \"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752\": rpc error: code = NotFound desc = could not find container \"f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752\": container with ID starting with f143eb146059e571262751d5298d677a492b85f7ee812ee051cc72de2fd38752 not found: ID does not exist"
Jul 28 23:25:15.157678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.157646    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:25:15.161165 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:15.161141    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68bb87f5d8-v952w"]
Jul 28 23:25:16.114192 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:16.114155    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93c5ea1d-9249-408c-aa6b-600a604cdc29" path="/var/lib/kubelet/pods/93c5ea1d-9249-408c-aa6b-600a604cdc29/volumes"
Jul 28 23:25:29.524960 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:29.524903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:25:29.574079 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:29.574043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:25:39.435170 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:25:39.435119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:12.476302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:12.476261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:12.580207 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:12.580173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:13.021062 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.021028    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"]
Jul 28 23:26:13.021446 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.021430    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93c5ea1d-9249-408c-aa6b-600a604cdc29" containerName="console"
Jul 28 23:26:13.021550 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.021449    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c5ea1d-9249-408c-aa6b-600a604cdc29" containerName="console"
Jul 28 23:26:13.021550 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.021522    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="93c5ea1d-9249-408c-aa6b-600a604cdc29" containerName="console"
Jul 28 23:26:13.024397 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.024377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.026808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.026790    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 23:26:13.026891 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.026853    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 23:26:13.027763 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.027749    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jg9h2\""
Jul 28 23:26:13.032576 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.032551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"]
Jul 28 23:26:13.160046 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.160007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.160228 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.160076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5xsz\" (UniqueName: \"kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.160228 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.160100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.260772 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.260739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.260961 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.260804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5xsz\" (UniqueName: \"kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.260961 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.260824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.261139 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.261117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.261201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.261138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.268785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.268760    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5xsz\" (UniqueName: \"kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.334955 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.334912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:13.464497 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:13.464468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"]
Jul 28 23:26:13.466440 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:26:13.466414    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80db3c6_02a1_4b71_9e5d_97b35a875907.slice/crio-2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba WatchSource:0}: Error finding container 2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba: Status 404 returned error can't find the container with id 2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba
Jul 28 23:26:14.374962 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:14.374921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g" event={"ID":"d80db3c6-02a1-4b71-9e5d-97b35a875907","Type":"ContainerStarted","Data":"2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba"}
Jul 28 23:26:20.403755 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:20.403666    2564 generic.go:356] "Generic (PLEG): container finished" podID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerID="9f101a96b5c07456d1db9ac3953ce1147f0dd2eff004017c18c51e4ed0a442ca" exitCode=0
Jul 28 23:26:20.404180 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:20.403754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g" event={"ID":"d80db3c6-02a1-4b71-9e5d-97b35a875907","Type":"ContainerDied","Data":"9f101a96b5c07456d1db9ac3953ce1147f0dd2eff004017c18c51e4ed0a442ca"}
Jul 28 23:26:23.418495 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:23.418461    2564 generic.go:356] "Generic (PLEG): container finished" podID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerID="6902fbae77674a5535414f4424a082ce04bbd1551edd706f42e7228416738aa6" exitCode=0
Jul 28 23:26:23.418903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:23.418540    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g" event={"ID":"d80db3c6-02a1-4b71-9e5d-97b35a875907","Type":"ContainerDied","Data":"6902fbae77674a5535414f4424a082ce04bbd1551edd706f42e7228416738aa6"}
Jul 28 23:26:30.454033 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:30.453998    2564 generic.go:356] "Generic (PLEG): container finished" podID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerID="9fc86423c8b6d5c12f8f0ec2df563579f94e9452aea8630bb9989d70ec2e2e4e" exitCode=0
Jul 28 23:26:30.454461 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:30.454041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g" event={"ID":"d80db3c6-02a1-4b71-9e5d-97b35a875907","Type":"ContainerDied","Data":"9fc86423c8b6d5c12f8f0ec2df563579f94e9452aea8630bb9989d70ec2e2e4e"}
Jul 28 23:26:31.586739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.586713    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:31.620371 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.620326    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util\") pod \"d80db3c6-02a1-4b71-9e5d-97b35a875907\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") "
Jul 28 23:26:31.620554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.620397    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5xsz\" (UniqueName: \"kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz\") pod \"d80db3c6-02a1-4b71-9e5d-97b35a875907\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") "
Jul 28 23:26:31.620554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.620434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle\") pod \"d80db3c6-02a1-4b71-9e5d-97b35a875907\" (UID: \"d80db3c6-02a1-4b71-9e5d-97b35a875907\") "
Jul 28 23:26:31.621296 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.621260    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle" (OuterVolumeSpecName: "bundle") pod "d80db3c6-02a1-4b71-9e5d-97b35a875907" (UID: "d80db3c6-02a1-4b71-9e5d-97b35a875907"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:26:31.622671 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.622636    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz" (OuterVolumeSpecName: "kube-api-access-s5xsz") pod "d80db3c6-02a1-4b71-9e5d-97b35a875907" (UID: "d80db3c6-02a1-4b71-9e5d-97b35a875907"). InnerVolumeSpecName "kube-api-access-s5xsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:26:31.624285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.624263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util" (OuterVolumeSpecName: "util") pod "d80db3c6-02a1-4b71-9e5d-97b35a875907" (UID: "d80db3c6-02a1-4b71-9e5d-97b35a875907"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:26:31.721384 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.721290    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:26:31.721384 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.721317    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5xsz\" (UniqueName: \"kubernetes.io/projected/d80db3c6-02a1-4b71-9e5d-97b35a875907-kube-api-access-s5xsz\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:26:31.721384 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:31.721333    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d80db3c6-02a1-4b71-9e5d-97b35a875907-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:26:32.464921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:32.464842    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g"
Jul 28 23:26:32.464921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:32.464866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407fb7g" event={"ID":"d80db3c6-02a1-4b71-9e5d-97b35a875907","Type":"ContainerDied","Data":"2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba"}
Jul 28 23:26:32.464921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:32.464921    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2abf8320356f9c12af6bc536ccf315d8d4ce68dd3b4272e550bfbba6906acdba"
Jul 28 23:26:33.989487 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:33.989454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:34.804053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:34.804013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:35.693452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693411    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"]
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693726    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="pull"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693736    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="pull"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693747    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="util"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693752    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="util"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693759    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="extract"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693765    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="extract"
Jul 28 23:26:35.693838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.693827    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d80db3c6-02a1-4b71-9e5d-97b35a875907" containerName="extract"
Jul 28 23:26:35.740460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.740424    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"]
Jul 28 23:26:35.740620 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.740520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.742985 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.742961    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:26:35.742985 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.742972    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 23:26:35.742985 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.742964    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-72vcg\""
Jul 28 23:26:35.856410 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.856373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245a6250-d462-4b95-b22e-a091c750c7a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.856410 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.856417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6dhm\" (UniqueName: \"kubernetes.io/projected/245a6250-d462-4b95-b22e-a091c750c7a4-kube-api-access-q6dhm\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.957294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.957211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245a6250-d462-4b95-b22e-a091c750c7a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.957294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.957248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6dhm\" (UniqueName: \"kubernetes.io/projected/245a6250-d462-4b95-b22e-a091c750c7a4-kube-api-access-q6dhm\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.957634 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.957612    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245a6250-d462-4b95-b22e-a091c750c7a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:35.965082 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:35.965052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6dhm\" (UniqueName: \"kubernetes.io/projected/245a6250-d462-4b95-b22e-a091c750c7a4-kube-api-access-q6dhm\") pod \"cert-manager-operator-controller-manager-55c444bf86-w9cn7\" (UID: \"245a6250-d462-4b95-b22e-a091c750c7a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:36.049757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:36.049719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"
Jul 28 23:26:36.211826 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:36.211749    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7"]
Jul 28 23:26:36.214431 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:26:36.214405    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245a6250_d462_4b95_b22e_a091c750c7a4.slice/crio-0b4cb8eb6af6edf2562a854514a7a246ee178114394be6ec92102ccf35d2078e WatchSource:0}: Error finding container 0b4cb8eb6af6edf2562a854514a7a246ee178114394be6ec92102ccf35d2078e: Status 404 returned error can't find the container with id 0b4cb8eb6af6edf2562a854514a7a246ee178114394be6ec92102ccf35d2078e
Jul 28 23:26:36.482377 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:36.482279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7" event={"ID":"245a6250-d462-4b95-b22e-a091c750c7a4","Type":"ContainerStarted","Data":"0b4cb8eb6af6edf2562a854514a7a246ee178114394be6ec92102ccf35d2078e"}
Jul 28 23:26:37.395277 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:37.395245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:37.448906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:37.448868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:39.262638 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.262602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:39.357213 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.357176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:39.457787 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.457753    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gh4nb"]
Jul 28 23:26:39.483850 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.483814    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gh4nb"]
Jul 28 23:26:39.483999 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.483909    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.487039 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.487012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ss64c\""
Jul 28 23:26:39.488226 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.488198    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 23:26:39.488356 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.488242    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 23:26:39.497887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.497863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7" event={"ID":"245a6250-d462-4b95-b22e-a091c750c7a4","Type":"ContainerStarted","Data":"8bf35bd94a1105138692a8d9e4ae18a39bb6c920d6be4409a3c2b838c3690fef"}
Jul 28 23:26:39.517491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.517403    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w9cn7" podStartSLOduration=2.200304264 podStartE2EDuration="4.51738682s" totalImagesPullingTime="2.317082556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:26:39.515116043 +0000 UTC m=+300.042165329" watchObservedRunningTime="2026-07-28 23:26:39.51738682 +0000 UTC m=+300.044436083"
Jul 28 23:26:39.590222 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.590188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.590457 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.590434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz7l9\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-kube-api-access-kz7l9\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.691100 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.691056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz7l9\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-kube-api-access-kz7l9\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.691100 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.691106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.707411 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.707388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.707553 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.707521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz7l9\" (UniqueName: \"kubernetes.io/projected/67a47a72-d02b-44c8-980d-26c544e19d6f-kube-api-access-kz7l9\") pod \"cert-manager-webhook-6486ffd997-gh4nb\" (UID: \"67a47a72-d02b-44c8-980d-26c544e19d6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.804504 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.804470    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:39.907275 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.907240    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:26:39.909297 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.909276    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:26:39.915882 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.915851    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:26:39.920055 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.920021    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:26:39.922932 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.922904    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 23:26:39.979215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:39.979167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gh4nb"]
Jul 28 23:26:39.982055 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:26:39.982028    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67a47a72_d02b_44c8_980d_26c544e19d6f.slice/crio-067afd21310ab651ff3566796751289ec6d1d46f9aecc3c5893ba1bdecc847c4 WatchSource:0}: Error finding container 067afd21310ab651ff3566796751289ec6d1d46f9aecc3c5893ba1bdecc847c4: Status 404 returned error can't find the container with id 067afd21310ab651ff3566796751289ec6d1d46f9aecc3c5893ba1bdecc847c4
Jul 28 23:26:40.093772 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:40.093579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:40.363336 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:40.180202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:40.503579 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:40.503542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb" event={"ID":"67a47a72-d02b-44c8-980d-26c544e19d6f","Type":"ContainerStarted","Data":"067afd21310ab651ff3566796751289ec6d1d46f9aecc3c5893ba1bdecc847c4"}
Jul 28 23:26:41.278018 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:41.277987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:41.668076 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:41.668042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:43.411720 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:43.411675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:44.394653 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:44.394620    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:44.449682 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:44.449603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:44.524391 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:44.524331    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb" event={"ID":"67a47a72-d02b-44c8-980d-26c544e19d6f","Type":"ContainerStarted","Data":"38a08b9621f9f483d8540d670d64a2d5174f2ed07e3b1cbab26ae427d151119b"}
Jul 28 23:26:44.524555 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:44.524410    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:44.539479 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:44.539428    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb" podStartSLOduration=1.375854474 podStartE2EDuration="5.539412981s" totalImagesPullingTime="4.163558507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:26:44.538875293 +0000 UTC m=+305.065924557" watchObservedRunningTime="2026-07-28 23:26:44.539412981 +0000 UTC m=+305.066462245"
Jul 28 23:26:45.579649 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:45.579618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:46.261158 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:46.261123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:46.362408 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:46.362375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:49.876453 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:49.876417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:26:50.015784 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.015746    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76qwt"]
Jul 28 23:26:50.019167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.019145    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.021548 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.021524    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-tjckx\""
Jul 28 23:26:50.025856 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.025828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76qwt"]
Jul 28 23:26:50.077925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.077882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.078128 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.077972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9fhb\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-kube-api-access-r9fhb\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.178992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.178910    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9fhb\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-kube-api-access-r9fhb\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.178992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.178983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.186780 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.186755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.186928 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.186907    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9fhb\" (UniqueName: \"kubernetes.io/projected/f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1-kube-api-access-r9fhb\") pod \"cert-manager-55b4d954d5-76qwt\" (UID: \"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1\") " pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.330481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.330444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-76qwt"
Jul 28 23:26:50.459806 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.459781    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76qwt"]
Jul 28 23:26:50.462614 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:26:50.462583    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9421ca5_e2b3_4f9d_95be_aa5c1bcab7e1.slice/crio-96ef2d278fac08624669933dc3f6831d1ce23b674e35a616845f6e836f4dc259 WatchSource:0}: Error finding container 96ef2d278fac08624669933dc3f6831d1ce23b674e35a616845f6e836f4dc259: Status 404 returned error can't find the container with id 96ef2d278fac08624669933dc3f6831d1ce23b674e35a616845f6e836f4dc259
Jul 28 23:26:50.532519 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.532481    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gh4nb"
Jul 28 23:26:50.553626 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.553589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-76qwt" event={"ID":"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1","Type":"ContainerStarted","Data":"3544ff3dd766086203ec2a40089c271c6a5855a702cf575746ffff860f2c65dd"}
Jul 28 23:26:50.553773 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.553643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-76qwt" event={"ID":"f9421ca5-e2b3-4f9d-95be-aa5c1bcab7e1","Type":"ContainerStarted","Data":"96ef2d278fac08624669933dc3f6831d1ce23b674e35a616845f6e836f4dc259"}
Jul 28 23:26:50.573451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:26:50.573394    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-76qwt" podStartSLOduration=0.573378851 podStartE2EDuration="573.378851ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:26:50.571825524 +0000 UTC m=+311.098874803" watchObservedRunningTime="2026-07-28 23:26:50.573378851 +0000 UTC m=+311.100428114"
Jul 28 23:27:08.402213 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:08.402168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:09.807809 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:09.807774    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:10.631148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.631105    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"]
Jul 28 23:27:10.634389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.634368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.636996 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.636973    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:27:10.637108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.637077    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 23:27:10.638392 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.638363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cz8sp\""
Jul 28 23:27:10.648373 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.645749    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"]
Jul 28 23:27:10.661182 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.661154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b37b1c8-7952-41f6-96f9-35b425a95622-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.661300 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.661203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkqhq\" (UniqueName: \"kubernetes.io/projected/5b37b1c8-7952-41f6-96f9-35b425a95622-kube-api-access-nkqhq\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.762557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.762522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkqhq\" (UniqueName: \"kubernetes.io/projected/5b37b1c8-7952-41f6-96f9-35b425a95622-kube-api-access-nkqhq\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.762728 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.762597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b37b1c8-7952-41f6-96f9-35b425a95622-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.762949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.762932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b37b1c8-7952-41f6-96f9-35b425a95622-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.770764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.770739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkqhq\" (UniqueName: \"kubernetes.io/projected/5b37b1c8-7952-41f6-96f9-35b425a95622-kube-api-access-nkqhq\") pod \"openshift-lws-operator-fd8ccff4c-bp2l2\" (UID: \"5b37b1c8-7952-41f6-96f9-35b425a95622\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:10.948016 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:10.947933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"
Jul 28 23:27:11.074723 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:11.074693    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2"]
Jul 28 23:27:11.076944 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:11.076914    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b37b1c8_7952_41f6_96f9_35b425a95622.slice/crio-4db882082d3dcaf2f8f2b470d90130467635570000b1bb25f763cb3141c64cce WatchSource:0}: Error finding container 4db882082d3dcaf2f8f2b470d90130467635570000b1bb25f763cb3141c64cce: Status 404 returned error can't find the container with id 4db882082d3dcaf2f8f2b470d90130467635570000b1bb25f763cb3141c64cce
Jul 28 23:27:11.078617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:11.078597    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:27:11.659653 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:11.659620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2" event={"ID":"5b37b1c8-7952-41f6-96f9-35b425a95622","Type":"ContainerStarted","Data":"4db882082d3dcaf2f8f2b470d90130467635570000b1bb25f763cb3141c64cce"}
Jul 28 23:27:16.687790 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:16.687759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2" event={"ID":"5b37b1c8-7952-41f6-96f9-35b425a95622","Type":"ContainerStarted","Data":"f40481dbe4259edc711ce093220cda36c5b04e832fb8aa1ec67ca2197608dc44"}
Jul 28 23:27:16.710692 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:16.705816    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bp2l2" podStartSLOduration=1.963375315 podStartE2EDuration="6.705788396s" totalImagesPullingTime="4.742413081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:16.703772029 +0000 UTC m=+337.230821292" watchObservedRunningTime="2026-07-28 23:27:16.705788396 +0000 UTC m=+337.232837658"
Jul 28 23:27:17.432249 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:17.432208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:17.529972 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:17.529934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:19.214661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:19.214624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:23.447281 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:23.447243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:23.535019 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:23.534956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:31.208196 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:31.208159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:31.304280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:31.304241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:31.826736 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:31.826700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.035297 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.035259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.136767 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.136687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.167182 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.167107    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"]
Jul 28 23:27:32.171980 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.171958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.174648 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.174624    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 23:27:32.174868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.174852    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 23:27:32.174919 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.174864    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 23:27:32.175317 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.175264    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7fb7f\""
Jul 28 23:27:32.175317 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.175282    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 23:27:32.185221 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.185190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"]
Jul 28 23:27:32.245242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.245200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.245717 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.245284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.245717 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.245322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.245717 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.245406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtmmr\" (UniqueName: \"kubernetes.io/projected/a76b2b36-e116-4399-bbb3-0d21e7971da4-kube-api-access-xtmmr\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.245717 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.245452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.335648 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.335613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.346763 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.346729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.346889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.346815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.346889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.346854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.346986 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.346914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtmmr\" (UniqueName: \"kubernetes.io/projected/a76b2b36-e116-4399-bbb3-0d21e7971da4-kube-api-access-xtmmr\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.346986 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.346945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.347336 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.347312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.348934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.348911    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a76b2b36-e116-4399-bbb3-0d21e7971da4-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.349278 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.349262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.349367 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.349326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a76b2b36-e116-4399-bbb3-0d21e7971da4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.355424 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.355403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtmmr\" (UniqueName: \"kubernetes.io/projected/a76b2b36-e116-4399-bbb3-0d21e7971da4-kube-api-access-xtmmr\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-pjvw2\" (UID: \"a76b2b36-e116-4399-bbb3-0d21e7971da4\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.439638 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.439555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.482658 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.482622    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:32.543831 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.543796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.622572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.622311    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"]
Jul 28 23:27:32.624503 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:32.624472    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76b2b36_e116_4399_bbb3_0d21e7971da4.slice/crio-51fa09af506b5cbd7961d0a825eed788f369cab3a39b178e4afd49131f9f5e5b WatchSource:0}: Error finding container 51fa09af506b5cbd7961d0a825eed788f369cab3a39b178e4afd49131f9f5e5b: Status 404 returned error can't find the container with id 51fa09af506b5cbd7961d0a825eed788f369cab3a39b178e4afd49131f9f5e5b
Jul 28 23:27:32.624639 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.624615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.684003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.683952    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:32.763683 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.763597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2" event={"ID":"a76b2b36-e116-4399-bbb3-0d21e7971da4","Type":"ContainerStarted","Data":"51fa09af506b5cbd7961d0a825eed788f369cab3a39b178e4afd49131f9f5e5b"}
Jul 28 23:27:32.782669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:32.782639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:36.644612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:36.644575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:36.746122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:36.746081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:36.783682 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:36.783645    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2" event={"ID":"a76b2b36-e116-4399-bbb3-0d21e7971da4","Type":"ContainerStarted","Data":"ac4d2b99a9bf6e3a2e7a9aa2b1d043e0be67b3143e879931a46ddecf80763394"}
Jul 28 23:27:37.789518 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:37.789481    2564 generic.go:356] "Generic (PLEG): container finished" podID="a76b2b36-e116-4399-bbb3-0d21e7971da4" containerID="ac4d2b99a9bf6e3a2e7a9aa2b1d043e0be67b3143e879931a46ddecf80763394" exitCode=0
Jul 28 23:27:37.789906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:37.789559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2" event={"ID":"a76b2b36-e116-4399-bbb3-0d21e7971da4","Type":"ContainerDied","Data":"ac4d2b99a9bf6e3a2e7a9aa2b1d043e0be67b3143e879931a46ddecf80763394"}
Jul 28 23:27:38.796813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:38.796777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2" event={"ID":"a76b2b36-e116-4399-bbb3-0d21e7971da4","Type":"ContainerStarted","Data":"37c8333eef2b2180b72ebce1c737c899de87b35826d62bd2eccff06b317026c7"}
Jul 28 23:27:38.797195 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:38.796884    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:38.817987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:38.817937    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2" podStartSLOduration=2.169137684 podStartE2EDuration="6.817921495s" totalImagesPullingTime="3.572920474s" totalInitContainerRuntime="1.075863337s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:38.815712362 +0000 UTC m=+359.342761633" watchObservedRunningTime="2026-07-28 23:27:38.817921495 +0000 UTC m=+359.344970757"
Jul 28 23:27:49.803975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:49.803933    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-pjvw2"
Jul 28 23:27:51.291183 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.291150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:51.394504 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.394472    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:51.703168 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.703084    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:51.815116 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.815082    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"]
Jul 28 23:27:51.836300 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.836266    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"]
Jul 28 23:27:51.836495 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.836443    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:51.838992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.838961    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 23:27:51.839134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.839080    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 23:27:51.839292 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.839274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z8jkr\""
Jul 28 23:27:51.924274 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.924240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrwj\" (UniqueName: \"kubernetes.io/projected/82d87be9-0d29-45cf-bb90-95ef38a03dde-kube-api-access-xsrwj\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:51.924274 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.924284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82d87be9-0d29-45cf-bb90-95ef38a03dde-tmp\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:51.924517 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.924467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:51.932153 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.932124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrpcd"]
Jul 28 23:27:51.935584 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.935563    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:51.938135 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.938114    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 23:27:51.938229 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.938154    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zvxcd\""
Jul 28 23:27:51.947932 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.947912    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrpcd"]
Jul 28 23:27:51.990779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:51.990693    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.025672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.025625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.025672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.025677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.025921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.025697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lnrt\" (UniqueName: \"kubernetes.io/projected/bf59ff25-6226-45cf-8c79-9002e905ff9c-kube-api-access-7lnrt\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.025921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.025735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrwj\" (UniqueName: \"kubernetes.io/projected/82d87be9-0d29-45cf-bb90-95ef38a03dde-kube-api-access-xsrwj\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.025921 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.025824    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 23:27:52.025921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.025866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82d87be9-0d29-45cf-bb90-95ef38a03dde-tmp\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.025921 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.025906    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs podName:82d87be9-0d29-45cf-bb90-95ef38a03dde nodeName:}" failed. No retries permitted until 2026-07-28 23:27:52.525881789 +0000 UTC m=+373.052931034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs") pod "kube-auth-proxy-77f8c77c69-9lspv" (UID: "82d87be9-0d29-45cf-bb90-95ef38a03dde") : secret "kube-auth-proxy-tls" not found
Jul 28 23:27:52.028185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.028160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82d87be9-0d29-45cf-bb90-95ef38a03dde-tmp\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.034563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.034536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrwj\" (UniqueName: \"kubernetes.io/projected/82d87be9-0d29-45cf-bb90-95ef38a03dde-kube-api-access-xsrwj\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.086572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.086534    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.126508 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.126471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.126676 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.126520    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lnrt\" (UniqueName: \"kubernetes.io/projected/bf59ff25-6226-45cf-8c79-9002e905ff9c-kube-api-access-7lnrt\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.126676 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.126632    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 23:27:52.126749 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.126695    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert podName:bf59ff25-6226-45cf-8c79-9002e905ff9c nodeName:}" failed. No retries permitted until 2026-07-28 23:27:52.626674876 +0000 UTC m=+373.153724117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert") pod "odh-model-controller-7998c67dc4-xrpcd" (UID: "bf59ff25-6226-45cf-8c79-9002e905ff9c") : secret "odh-model-controller-webhook-cert" not found
Jul 28 23:27:52.137520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.137489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lnrt\" (UniqueName: \"kubernetes.io/projected/bf59ff25-6226-45cf-8c79-9002e905ff9c-kube-api-access-7lnrt\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.186966 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.186929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.308631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.308596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.406871 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.406839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.530217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.530173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.532672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.532642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d87be9-0d29-45cf-bb90-95ef38a03dde-tls-certs\") pod \"kube-auth-proxy-77f8c77c69-9lspv\" (UID: \"82d87be9-0d29-45cf-bb90-95ef38a03dde\") " pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.631709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.631624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.633941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.633913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf59ff25-6226-45cf-8c79-9002e905ff9c-cert\") pod \"odh-model-controller-7998c67dc4-xrpcd\" (UID: \"bf59ff25-6226-45cf-8c79-9002e905ff9c\") " pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.686882 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.686846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.701874 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.701838    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lnv9v"]
Jul 28 23:27:52.705685 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.705665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.708531 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.708510    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 23:27:52.708651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.708514    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-x885f\""
Jul 28 23:27:52.715959 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.715927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lnv9v"]
Jul 28 23:27:52.746954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.746922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"
Jul 28 23:27:52.834009 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.833982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.834134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.834071    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqwc\" (UniqueName: \"kubernetes.io/projected/5be4c05b-ebc6-46b6-8945-1fa180e0a124-kube-api-access-gpqwc\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.841705 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.841680    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:27:52.843405 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.843377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.846263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.846242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:52.846557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.846541    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.849465 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.849260    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 23:27:52.849465 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.849324    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 23:27:52.849710 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.849680    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 23:27:52.849855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.849686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 23:27:52.849974 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.849958    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5qfmt\""
Jul 28 23:27:52.859256 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.859158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:27:52.920992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.920965    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv"]
Jul 28 23:27:52.922870 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:52.922839    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d87be9_0d29_45cf_bb90_95ef38a03dde.slice/crio-71acd378f73102210facd760ef5ebf913f1c2eb3c86aef2b8832f285939fbfa9 WatchSource:0}: Error finding container 71acd378f73102210facd760ef5ebf913f1c2eb3c86aef2b8832f285939fbfa9: Status 404 returned error can't find the container with id 71acd378f73102210facd760ef5ebf913f1c2eb3c86aef2b8832f285939fbfa9
Jul 28 23:27:52.934700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.934807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqwc\" (UniqueName: \"kubernetes.io/projected/5be4c05b-ebc6-46b6-8945-1fa180e0a124-kube-api-access-gpqwc\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.934807 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.934907 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjvh\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.934960 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.935003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.935003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.934992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.935071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.935017    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.935071 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.935043    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 23:27:52.935141 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:27:52.935103    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs podName:5be4c05b-ebc6-46b6-8945-1fa180e0a124 nodeName:}" failed. No retries permitted until 2026-07-28 23:27:53.435088417 +0000 UTC m=+373.962137661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs") pod "model-serving-api-54d89bd5fc-lnv9v" (UID: "5be4c05b-ebc6-46b6-8945-1fa180e0a124") : secret "model-serving-api-tls" not found
Jul 28 23:27:52.935141 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.935047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:52.952487 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.952451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqwc\" (UniqueName: \"kubernetes.io/projected/5be4c05b-ebc6-46b6-8945-1fa180e0a124-kube-api-access-gpqwc\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:52.958377 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.958330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:52.990068 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:52.990015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xrpcd"]
Jul 28 23:27:52.992140 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:52.992115    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf59ff25_6226_45cf_8c79_9002e905ff9c.slice/crio-696b8b634dc9c0130c9235f63c32480bacf8bbc5b7ce777766d2f0405c84dae4 WatchSource:0}: Error finding container 696b8b634dc9c0130c9235f63c32480bacf8bbc5b7ce777766d2f0405c84dae4: Status 404 returned error can't find the container with id 696b8b634dc9c0130c9235f63c32480bacf8bbc5b7ce777766d2f0405c84dae4
Jul 28 23:27:53.036166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjvh\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036483 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036621 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036782 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036839 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.036898 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.036863    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.037166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.037146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.038761 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.038739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.039212 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.039183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.039303 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.039207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.039430 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.039414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.043742 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.043722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjvh\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.044022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.043999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-srf4f\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.074520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.074486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:53.159294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.159211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:53.292185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.292126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:27:53.294279 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:53.294243    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a91ba6_d425_4577_9188_1a0ead92c33e.slice/crio-ffc5a436e353611118b95e006a8e2ac1234677bbcccc114c9b51d450536dadc9 WatchSource:0}: Error finding container ffc5a436e353611118b95e006a8e2ac1234677bbcccc114c9b51d450536dadc9: Status 404 returned error can't find the container with id ffc5a436e353611118b95e006a8e2ac1234677bbcccc114c9b51d450536dadc9
Jul 28 23:27:53.441067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.440990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:53.444597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.444566    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5be4c05b-ebc6-46b6-8945-1fa180e0a124-tls-certs\") pod \"model-serving-api-54d89bd5fc-lnv9v\" (UID: \"5be4c05b-ebc6-46b6-8945-1fa180e0a124\") " pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:53.617871 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.617834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:53.791058 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.790956    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lnv9v"]
Jul 28 23:27:53.794806 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:27:53.794766    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be4c05b_ebc6_46b6_8945_1fa180e0a124.slice/crio-413b7a1416e7bf2ba3be1e275073beefbe8e16b4009fe1b9bec641010510d0a1 WatchSource:0}: Error finding container 413b7a1416e7bf2ba3be1e275073beefbe8e16b4009fe1b9bec641010510d0a1: Status 404 returned error can't find the container with id 413b7a1416e7bf2ba3be1e275073beefbe8e16b4009fe1b9bec641010510d0a1
Jul 28 23:27:53.843127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.843049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:53.879813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.879677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v" event={"ID":"5be4c05b-ebc6-46b6-8945-1fa180e0a124","Type":"ContainerStarted","Data":"413b7a1416e7bf2ba3be1e275073beefbe8e16b4009fe1b9bec641010510d0a1"}
Jul 28 23:27:53.881552 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.881468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv" event={"ID":"82d87be9-0d29-45cf-bb90-95ef38a03dde","Type":"ContainerStarted","Data":"71acd378f73102210facd760ef5ebf913f1c2eb3c86aef2b8832f285939fbfa9"}
Jul 28 23:27:53.883538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.883506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" event={"ID":"79a91ba6-d425-4577-9188-1a0ead92c33e","Type":"ContainerStarted","Data":"ffc5a436e353611118b95e006a8e2ac1234677bbcccc114c9b51d450536dadc9"}
Jul 28 23:27:53.886275 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:53.886217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd" event={"ID":"bf59ff25-6226-45cf-8c79-9002e905ff9c","Type":"ContainerStarted","Data":"696b8b634dc9c0130c9235f63c32480bacf8bbc5b7ce777766d2f0405c84dae4"}
Jul 28 23:27:57.918949 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.918873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd" event={"ID":"bf59ff25-6226-45cf-8c79-9002e905ff9c","Type":"ContainerStarted","Data":"ec84d0a3266ea0088d741c3f4d1932a8d534ff80c29a7e28d9dd26c9d34b19ae"}
Jul 28 23:27:57.919433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.919023    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:27:57.920606 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.920578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v" event={"ID":"5be4c05b-ebc6-46b6-8945-1fa180e0a124","Type":"ContainerStarted","Data":"988b40a6492b6c14a486b77740b0c58ba5729b15461833da271c3e6a6cb3bedd"}
Jul 28 23:27:57.920721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.920633    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:27:57.922110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.922086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv" event={"ID":"82d87be9-0d29-45cf-bb90-95ef38a03dde","Type":"ContainerStarted","Data":"be9c7f9ce262a86d1df21cb1d86d4c8ea19e4e81d9f8a9eaecbbf7a0ad16abf2"}
Jul 28 23:27:57.936230 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.936180    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd" podStartSLOduration=2.606195633 podStartE2EDuration="6.936164307s" totalImagesPullingTime="4.329968674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:57.934095697 +0000 UTC m=+378.461144960" watchObservedRunningTime="2026-07-28 23:27:57.936164307 +0000 UTC m=+378.463213569"
Jul 28 23:27:57.950079 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.950019    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77f8c77c69-9lspv" podStartSLOduration=2.594873851 podStartE2EDuration="6.950002393s" totalImagesPullingTime="4.355128542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:57.948917365 +0000 UTC m=+378.475966653" watchObservedRunningTime="2026-07-28 23:27:57.950002393 +0000 UTC m=+378.477051655"
Jul 28 23:27:57.967000 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:57.966940    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v" podStartSLOduration=2.450691769 podStartE2EDuration="5.966918211s" totalImagesPullingTime="3.516226442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:57.964455898 +0000 UTC m=+378.491505161" watchObservedRunningTime="2026-07-28 23:27:57.966918211 +0000 UTC m=+378.493967477"
Jul 28 23:27:58.654181 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:58.654145    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:27:58.654250 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:58.654221    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:27:58.929472 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:58.929381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" event={"ID":"79a91ba6-d425-4577-9188-1a0ead92c33e","Type":"ContainerStarted","Data":"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"}
Jul 28 23:27:58.929886 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:58.929554    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:27:58.949655 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:58.949606    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" podStartSLOduration=1.592067486 podStartE2EDuration="6.949591773s" totalImagesPullingTime="5.357524287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:58.94796353 +0000 UTC m=+379.475012795" watchObservedRunningTime="2026-07-28 23:27:58.949591773 +0000 UTC m=+379.476641036"
Jul 28 23:27:59.259862 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.259768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:59.368441 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.368406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:59.929128 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.929084    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"]
Jul 28 23:27:59.933669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.933630    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:27:59.937188 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.937159    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-782r6\""
Jul 28 23:27:59.940529 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.940440    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-srf4f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 23:27:59.940529 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.940504    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 23:27:59.955215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.955139    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:27:59.956011 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:27:59.955983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"]
Jul 28 23:28:00.010521 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010521 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010923 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.010982 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.010947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdz2t\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-kube-api-access-tdz2t\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.011056 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.011031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.011113 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.011075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112017 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.111986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112017 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdz2t\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-kube-api-access-tdz2t\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112542 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112542 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.112832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.112813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.113390 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.113334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.114518 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.114495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.114825 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.114804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.121903 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.121877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdz2t\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-kube-api-access-tdz2t\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.123730 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.123710    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm5xph\" (UID: \"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.245464 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.245381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:00.253982 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.253954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:00.420621 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.420552    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"]
Jul 28 23:28:00.421712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.421681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:00.423581 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:00.423553    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b721ebe_c34e_4205_ac9a_d6aeb5e0a11e.slice/crio-ca4f771d7c0e7769491f51a5672fc2fc1d63d10ae79311674234aa51075729b9 WatchSource:0}: Error finding container ca4f771d7c0e7769491f51a5672fc2fc1d63d10ae79311674234aa51075729b9: Status 404 returned error can't find the container with id ca4f771d7c0e7769491f51a5672fc2fc1d63d10ae79311674234aa51075729b9
Jul 28 23:28:00.944794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:00.944759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph" event={"ID":"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e","Type":"ContainerStarted","Data":"ca4f771d7c0e7769491f51a5672fc2fc1d63d10ae79311674234aa51075729b9"}
Jul 28 23:28:01.308801 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.308767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:01.362189 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.362094    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"]
Jul 28 23:28:01.366121 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.366096    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.368712 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.368689    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 23:28:01.369447 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.369425    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ml94k\""
Jul 28 23:28:01.374314 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.374221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"]
Jul 28 23:28:01.380089 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.380063    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"]
Jul 28 23:28:01.383986 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.383964    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.385291 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.385224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:01.387832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.386529    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fhc87\""
Jul 28 23:28:01.387832 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.386415    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 23:28:01.394806 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.394783    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"]
Jul 28 23:28:01.426093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.426053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9774aa96-fd1c-4820-9018-271302c43913-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.426254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.426183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr5nr\" (UniqueName: \"kubernetes.io/projected/9774aa96-fd1c-4820-9018-271302c43913-kube-api-access-nr5nr\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.426330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.426269    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jt6c\" (UniqueName: \"kubernetes.io/projected/e62753f4-f908-4cc4-a6f1-293529dd0665-kube-api-access-2jt6c\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.426330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.426321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.519138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.519096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:01.527416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.527379    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr5nr\" (UniqueName: \"kubernetes.io/projected/9774aa96-fd1c-4820-9018-271302c43913-kube-api-access-nr5nr\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.527574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.527422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jt6c\" (UniqueName: \"kubernetes.io/projected/e62753f4-f908-4cc4-a6f1-293529dd0665-kube-api-access-2jt6c\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.527574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.527474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.527574 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.527509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9774aa96-fd1c-4820-9018-271302c43913-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.527739 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:28:01.527707    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 28 23:28:01.527798 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:28:01.527771    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert podName:e62753f4-f908-4cc4-a6f1-293529dd0665 nodeName:}" failed. No retries permitted until 2026-07-28 23:28:02.027751303 +0000 UTC m=+382.554800550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert") pod "kserve-controller-manager-d6f4b65b8-zsgtr" (UID: "e62753f4-f908-4cc4-a6f1-293529dd0665") : secret "kserve-webhook-server-cert" not found
Jul 28 23:28:01.530808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.530783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9774aa96-fd1c-4820-9018-271302c43913-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.540880 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.540855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr5nr\" (UniqueName: \"kubernetes.io/projected/9774aa96-fd1c-4820-9018-271302c43913-kube-api-access-nr5nr\") pod \"llmisvc-controller-manager-777c6bd7d5-bprkh\" (UID: \"9774aa96-fd1c-4820-9018-271302c43913\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.541042 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.541020    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jt6c\" (UniqueName: \"kubernetes.io/projected/e62753f4-f908-4cc4-a6f1-293529dd0665-kube-api-access-2jt6c\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:01.670587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.670494    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:01.699843 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.699803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:01.853583 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.853546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"]
Jul 28 23:28:01.856338 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:01.856310    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9774aa96_fd1c_4820_9018_271302c43913.slice/crio-7fcfcf72c9045441c2b7c0769960729819d437cb2da12d793cc79bdd3e35c11a WatchSource:0}: Error finding container 7fcfcf72c9045441c2b7c0769960729819d437cb2da12d793cc79bdd3e35c11a: Status 404 returned error can't find the container with id 7fcfcf72c9045441c2b7c0769960729819d437cb2da12d793cc79bdd3e35c11a
Jul 28 23:28:01.952557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:01.952456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh" event={"ID":"9774aa96-fd1c-4820-9018-271302c43913","Type":"ContainerStarted","Data":"7fcfcf72c9045441c2b7c0769960729819d437cb2da12d793cc79bdd3e35c11a"}
Jul 28 23:28:02.032366 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.032305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:02.035017 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.034992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e62753f4-f908-4cc4-a6f1-293529dd0665-cert\") pod \"kserve-controller-manager-d6f4b65b8-zsgtr\" (UID: \"e62753f4-f908-4cc4-a6f1-293529dd0665\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:02.281335 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.281243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:02.575664 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.575619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:02.674361 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.674313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:02.773917 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.773879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:02.939879 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:02.939798    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:28:03.324522 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.324487    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:28:03.324826 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.324565    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:28:03.324826 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.324597    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:28:03.359800 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.359766    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:28:03.360014 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.359986    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerName="discovery" containerID="cri-o://1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d" gracePeriod=30
Jul 28 23:28:03.421565 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.421540    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"]
Jul 28 23:28:03.454272 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:03.454240    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62753f4_f908_4cc4_a6f1_293529dd0665.slice/crio-5c8c13fb4f3ee651d508b2966f24a0284369ee780ad680c16afcb3f8e0aa37d4 WatchSource:0}: Error finding container 5c8c13fb4f3ee651d508b2966f24a0284369ee780ad680c16afcb3f8e0aa37d4: Status 404 returned error can't find the container with id 5c8c13fb4f3ee651d508b2966f24a0284369ee780ad680c16afcb3f8e0aa37d4
Jul 28 23:28:03.635895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.635858    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:28:03.649833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.649796    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.649997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.649859    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.649997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.649965    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.650127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.650004    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmjvh\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.650127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.650039    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.650127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.650070    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.650127 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.650099    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig\") pod \"79a91ba6-d425-4577-9188-1a0ead92c33e\" (UID: \"79a91ba6-d425-4577-9188-1a0ead92c33e\") "
Jul 28 23:28:03.650538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.650282    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:28:03.653115 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.653081    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token" (OuterVolumeSpecName: "istio-token") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:28:03.654484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.654241    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts" (OuterVolumeSpecName: "cacerts") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:28:03.655317 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.655241    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh" (OuterVolumeSpecName: "kube-api-access-fmjvh") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "kube-api-access-fmjvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:28:03.655823 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.655482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:28:03.655823 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.655573    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs" (OuterVolumeSpecName: "local-certs") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:28:03.657975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.657947    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "79a91ba6-d425-4577-9188-1a0ead92c33e" (UID: "79a91ba6-d425-4577-9188-1a0ead92c33e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:28:03.751830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751794    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.751830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751831    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-ca-configmap\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.752077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751846    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-csr-dns-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.752077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751860    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmjvh\" (UniqueName: \"kubernetes.io/projected/79a91ba6-d425-4577-9188-1a0ead92c33e-kube-api-access-fmjvh\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.752077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751875    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-cacerts\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.752077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751889    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a91ba6-d425-4577-9188-1a0ead92c33e-local-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.752077 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.751901    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79a91ba6-d425-4577-9188-1a0ead92c33e-istio-kubeconfig\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:03.967224 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.967138    2564 generic.go:356] "Generic (PLEG): container finished" podID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerID="1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d" exitCode=0
Jul 28 23:28:03.967224 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.967184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" event={"ID":"79a91ba6-d425-4577-9188-1a0ead92c33e","Type":"ContainerDied","Data":"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"}
Jul 28 23:28:03.967224 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.967210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"
Jul 28 23:28:03.967547 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.967242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f" event={"ID":"79a91ba6-d425-4577-9188-1a0ead92c33e","Type":"ContainerDied","Data":"ffc5a436e353611118b95e006a8e2ac1234677bbcccc114c9b51d450536dadc9"}
Jul 28 23:28:03.967547 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.967270    2564 scope.go:117] "RemoveContainer" containerID="1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"
Jul 28 23:28:03.971378 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.970395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph" event={"ID":"3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e","Type":"ContainerStarted","Data":"1b5a50293ce04457ac0eaeadad2e7fd63249edc4da2950647d810ba37bf74759"}
Jul 28 23:28:03.973256 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:03.973229    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr" event={"ID":"e62753f4-f908-4cc4-a6f1-293529dd0665","Type":"ContainerStarted","Data":"5c8c13fb4f3ee651d508b2966f24a0284369ee780ad680c16afcb3f8e0aa37d4"}
Jul 28 23:28:04.032594 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.032515    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph" podStartSLOduration=2.134032391 podStartE2EDuration="5.032493388s" totalImagesPullingTime="2.898460997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:04.030833425 +0000 UTC m=+384.557882688" watchObservedRunningTime="2026-07-28 23:28:04.032493388 +0000 UTC m=+384.559542654"
Jul 28 23:28:04.057432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.057395    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:28:04.058409 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.058387    2564 scope.go:117] "RemoveContainer" containerID="1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"
Jul 28 23:28:04.058792 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:28:04.058766    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d\": container with ID starting with 1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d not found: ID does not exist" containerID="1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"
Jul 28 23:28:04.058869 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.058804    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d"} err="failed to get container status \"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d\": rpc error: code = NotFound desc = could not find container \"1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d\": container with ID starting with 1bcfe00625f5a1d6078f05edde88db59bd9609aa468d3130b407125f5f5e042d not found: ID does not exist"
Jul 28 23:28:04.074414 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.074377    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-srf4f"]
Jul 28 23:28:04.116871 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.116837    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" path="/var/lib/kubelet/pods/79a91ba6-d425-4577-9188-1a0ead92c33e/volumes"
Jul 28 23:28:04.254658 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.254576    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:04.259987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.259953    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:04.980172 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.980141    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:04.981098 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:04.981080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm5xph"
Jul 28 23:28:06.180040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.179998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:06.231380 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.230563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:06.992857 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.992816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr" event={"ID":"e62753f4-f908-4cc4-a6f1-293529dd0665","Type":"ContainerStarted","Data":"2940c04466dee00d3e49e3935dd6cf3f5342e175ee717d89d62156182b7d02cd"}
Jul 28 23:28:06.993066 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.992923    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:06.994395 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.994363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh" event={"ID":"9774aa96-fd1c-4820-9018-271302c43913","Type":"ContainerStarted","Data":"92c2bfb7e2e83f6c8873c92d0548593236c6f724d2d70f8559c133985b35ac6d"}
Jul 28 23:28:06.994576 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:06.994559    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:07.009338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:07.009290    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr" podStartSLOduration=3.146045734 podStartE2EDuration="6.009276097s" totalImagesPullingTime="2.863230363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:07.007338886 +0000 UTC m=+387.534388149" watchObservedRunningTime="2026-07-28 23:28:07.009276097 +0000 UTC m=+387.536325397"
Jul 28 23:28:07.024484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:07.024434    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh" podStartSLOduration=1.557082785 podStartE2EDuration="6.024419461s" totalImagesPullingTime="4.467336676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:07.022683403 +0000 UTC m=+387.549732664" watchObservedRunningTime="2026-07-28 23:28:07.024419461 +0000 UTC m=+387.551468723"
Jul 28 23:28:08.932165 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:08.932132    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xrpcd"
Jul 28 23:28:08.934056 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:08.934033    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lnv9v"
Jul 28 23:28:38.001636 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:38.001605    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bprkh"
Jul 28 23:28:38.004702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:38.004674    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zsgtr"
Jul 28 23:28:42.850543 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.850503    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58997c4d99-88rlb"]
Jul 28 23:28:42.851120 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.851098    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerName="discovery"
Jul 28 23:28:42.851216 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.851123    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerName="discovery"
Jul 28 23:28:42.851216 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.851208    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a91ba6-d425-4577-9188-1a0ead92c33e" containerName="discovery"
Jul 28 23:28:42.857777 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.857751    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:42.864700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.864668    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58997c4d99-88rlb"]
Jul 28 23:28:42.895085 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:42.895028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:43.017535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-service-ca\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-trusted-ca-bundle\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-oauth-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b966d\" (UniqueName: \"kubernetes.io/projected/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-kube-api-access-b966d\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-oauth-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.017976 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.017811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118336 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-trusted-ca-bundle\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118336 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-oauth-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118336 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b966d\" (UniqueName: \"kubernetes.io/projected/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-kube-api-access-b966d\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-oauth-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.118665 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.118472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-service-ca\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.119238 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.119206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-oauth-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.119325 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.119211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-service-ca\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.119325 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.119264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.119325 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.119305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-trusted-ca-bundle\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.120852 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.120831    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-oauth-config\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.120965 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.120947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-console-serving-cert\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.125733 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.125715    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b966d\" (UniqueName: \"kubernetes.io/projected/62c178a4-085d-4fcb-80d1-cfadfd6f2faf-kube-api-access-b966d\") pod \"console-58997c4d99-88rlb\" (UID: \"62c178a4-085d-4fcb-80d1-cfadfd6f2faf\") " pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.169326 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.169294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:43.304569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:43.304542    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58997c4d99-88rlb"]
Jul 28 23:28:43.306628 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:43.306601    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c178a4_085d_4fcb_80d1_cfadfd6f2faf.slice/crio-df9d7fd34d1bd246358e5c4182f70cf2badda2f6393d81f4ed96a2a050599469 WatchSource:0}: Error finding container df9d7fd34d1bd246358e5c4182f70cf2badda2f6393d81f4ed96a2a050599469: Status 404 returned error can't find the container with id df9d7fd34d1bd246358e5c4182f70cf2badda2f6393d81f4ed96a2a050599469
Jul 28 23:28:44.172394 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:44.172328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58997c4d99-88rlb" event={"ID":"62c178a4-085d-4fcb-80d1-cfadfd6f2faf","Type":"ContainerStarted","Data":"4646b670e69fe9eba951e2143753a9741f2f81712df4934f3c80208242857b1c"}
Jul 28 23:28:44.172845 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:44.172416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58997c4d99-88rlb" event={"ID":"62c178a4-085d-4fcb-80d1-cfadfd6f2faf","Type":"ContainerStarted","Data":"df9d7fd34d1bd246358e5c4182f70cf2badda2f6393d81f4ed96a2a050599469"}
Jul 28 23:28:44.190486 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:44.190430    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58997c4d99-88rlb" podStartSLOduration=2.190414754 podStartE2EDuration="2.190414754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:44.188514101 +0000 UTC m=+424.715563391" watchObservedRunningTime="2026-07-28 23:28:44.190414754 +0000 UTC m=+424.717464017"
Jul 28 23:28:52.740377 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.740329    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:52.872107 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.872060    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:52.876007 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.875984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:52.878490 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.878463    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 23:28:52.878490 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.878478    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 23:28:52.878696 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.878478    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6bkbr\""
Jul 28 23:28:52.885277 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.885253    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:52.908268 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.908239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6skwq\" (UniqueName: \"kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq\") pod \"kuadrant-operator-catalog-vhf7d\" (UID: \"5e4c7822-6e04-4e05-b400-60e22472f7e9\") " pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:52.934376 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:52.934318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.009281 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.009185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6skwq\" (UniqueName: \"kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq\") pod \"kuadrant-operator-catalog-vhf7d\" (UID: \"5e4c7822-6e04-4e05-b400-60e22472f7e9\") " pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:53.016938 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.016907    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6skwq\" (UniqueName: \"kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq\") pod \"kuadrant-operator-catalog-vhf7d\" (UID: \"5e4c7822-6e04-4e05-b400-60e22472f7e9\") " pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:53.032518 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.032492    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.170285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.170251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:53.170469 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.170309    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:53.175597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.175568    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:53.189096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.189060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:53.223785 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.223749    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58997c4d99-88rlb"
Jul 28 23:28:53.233506 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.233474    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:53.274587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.274440    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:28:53.357808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.357782    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:53.360054 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:53.360021    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4c7822_6e04_4e05_b400_60e22472f7e9.slice/crio-fe2ef8fe7e7d2a919c270da549c3c66437f13d32037aead8281ac8614cd41264 WatchSource:0}: Error finding container fe2ef8fe7e7d2a919c270da549c3c66437f13d32037aead8281ac8614cd41264: Status 404 returned error can't find the container with id fe2ef8fe7e7d2a919c270da549c3c66437f13d32037aead8281ac8614cd41264
Jul 28 23:28:53.418611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.418565    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.441139 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.441105    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n2zv7"]
Jul 28 23:28:53.445794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.445777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:28:53.450818 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.450792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n2zv7"]
Jul 28 23:28:53.514387 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.514333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjq8\" (UniqueName: \"kubernetes.io/projected/10e14de4-2c8e-4165-ac71-f608c25cdcfb-kube-api-access-rpjq8\") pod \"kuadrant-operator-catalog-n2zv7\" (UID: \"10e14de4-2c8e-4165-ac71-f608c25cdcfb\") " pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:28:53.523575 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.523546    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.575909 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.575874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.615023 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.614988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjq8\" (UniqueName: \"kubernetes.io/projected/10e14de4-2c8e-4165-ac71-f608c25cdcfb-kube-api-access-rpjq8\") pod \"kuadrant-operator-catalog-n2zv7\" (UID: \"10e14de4-2c8e-4165-ac71-f608c25cdcfb\") " pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:28:53.622978 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.622953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjq8\" (UniqueName: \"kubernetes.io/projected/10e14de4-2c8e-4165-ac71-f608c25cdcfb-kube-api-access-rpjq8\") pod \"kuadrant-operator-catalog-n2zv7\" (UID: \"10e14de4-2c8e-4165-ac71-f608c25cdcfb\") " pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:28:53.668867 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.668829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.757296 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.757260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:28:53.865020 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.864982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:28:53.903406 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:53.903383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n2zv7"]
Jul 28 23:28:53.905242 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:28:53.905213    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e14de4_2c8e_4165_ac71_f608c25cdcfb.slice/crio-bcc7ceb8132460a498ca7299697003e497775109bce7298fc59a5cbe9a021676 WatchSource:0}: Error finding container bcc7ceb8132460a498ca7299697003e497775109bce7298fc59a5cbe9a021676: Status 404 returned error can't find the container with id bcc7ceb8132460a498ca7299697003e497775109bce7298fc59a5cbe9a021676
Jul 28 23:28:54.226245 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:54.226156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7" event={"ID":"10e14de4-2c8e-4165-ac71-f608c25cdcfb","Type":"ContainerStarted","Data":"bcc7ceb8132460a498ca7299697003e497775109bce7298fc59a5cbe9a021676"}
Jul 28 23:28:54.227685 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:54.227640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" event={"ID":"5e4c7822-6e04-4e05-b400-60e22472f7e9","Type":"ContainerStarted","Data":"fe2ef8fe7e7d2a919c270da549c3c66437f13d32037aead8281ac8614cd41264"}
Jul 28 23:28:56.241914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.241814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7" event={"ID":"10e14de4-2c8e-4165-ac71-f608c25cdcfb","Type":"ContainerStarted","Data":"38c9ce44c78edf93ab3c4762a1361ac0341d3ef951dfd78e086c9e33a5f80109"}
Jul 28 23:28:56.243356 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.243316    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" event={"ID":"5e4c7822-6e04-4e05-b400-60e22472f7e9","Type":"ContainerStarted","Data":"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"}
Jul 28 23:28:56.243479 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.243402    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" podUID="5e4c7822-6e04-4e05-b400-60e22472f7e9" containerName="registry-server" containerID="cri-o://40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1" gracePeriod=2
Jul 28 23:28:56.256674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.256629    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7" podStartSLOduration=1.237250343 podStartE2EDuration="3.256615126s" totalImagesPullingTime="2.019364783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:56.255455724 +0000 UTC m=+436.782504987" watchObservedRunningTime="2026-07-28 23:28:56.256615126 +0000 UTC m=+436.783664389"
Jul 28 23:28:56.268961 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.268869    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" podStartSLOduration=1.700745764 podStartE2EDuration="4.268851019s" totalImagesPullingTime="2.568105255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:56.268745062 +0000 UTC m=+436.795794325" watchObservedRunningTime="2026-07-28 23:28:56.268851019 +0000 UTC m=+436.795900283"
Jul 28 23:28:56.496366 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.496285    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:56.544276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.544239    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6skwq\" (UniqueName: \"kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq\") pod \"5e4c7822-6e04-4e05-b400-60e22472f7e9\" (UID: \"5e4c7822-6e04-4e05-b400-60e22472f7e9\") "
Jul 28 23:28:56.546440 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.546417    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq" (OuterVolumeSpecName: "kube-api-access-6skwq") pod "5e4c7822-6e04-4e05-b400-60e22472f7e9" (UID: "5e4c7822-6e04-4e05-b400-60e22472f7e9"). InnerVolumeSpecName "kube-api-access-6skwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:28:56.645920 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:56.645877    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6skwq\" (UniqueName: \"kubernetes.io/projected/5e4c7822-6e04-4e05-b400-60e22472f7e9-kube-api-access-6skwq\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:28:57.250727 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.250630    2564 generic.go:356] "Generic (PLEG): container finished" podID="5e4c7822-6e04-4e05-b400-60e22472f7e9" containerID="40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1" exitCode=0
Jul 28 23:28:57.250727 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.250693    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d"
Jul 28 23:28:57.251227 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.250718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" event={"ID":"5e4c7822-6e04-4e05-b400-60e22472f7e9","Type":"ContainerDied","Data":"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"}
Jul 28 23:28:57.251227 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.250792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vhf7d" event={"ID":"5e4c7822-6e04-4e05-b400-60e22472f7e9","Type":"ContainerDied","Data":"fe2ef8fe7e7d2a919c270da549c3c66437f13d32037aead8281ac8614cd41264"}
Jul 28 23:28:57.251227 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.250819    2564 scope.go:117] "RemoveContainer" containerID="40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"
Jul 28 23:28:57.260213 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.260188    2564 scope.go:117] "RemoveContainer" containerID="40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"
Jul 28 23:28:57.260584 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:28:57.260561    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1\": container with ID starting with 40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1 not found: ID does not exist" containerID="40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"
Jul 28 23:28:57.260648 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.260594    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1"} err="failed to get container status \"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1\": rpc error: code = NotFound desc = could not find container \"40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1\": container with ID starting with 40477e74ed007edc526176f0180e5a727e191c3c64edee590a25c9d576473be1 not found: ID does not exist"
Jul 28 23:28:57.271770 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.271749    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:57.274991 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:57.274969    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vhf7d"]
Jul 28 23:28:58.115246 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:28:58.115209    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e4c7822-6e04-4e05-b400-60e22472f7e9" path="/var/lib/kubelet/pods/5e4c7822-6e04-4e05-b400-60e22472f7e9/volumes"
Jul 28 23:29:03.757575 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:03.757489    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:29:03.757575 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:03.757553    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:29:03.779657 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:03.779632    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:29:04.307776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:04.307747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-n2zv7"
Jul 28 23:29:08.070611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.070569    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"]
Jul 28 23:29:08.071148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.071116    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e4c7822-6e04-4e05-b400-60e22472f7e9" containerName="registry-server"
Jul 28 23:29:08.071148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.071139    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e4c7822-6e04-4e05-b400-60e22472f7e9" containerName="registry-server"
Jul 28 23:29:08.071252 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.071233    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e4c7822-6e04-4e05-b400-60e22472f7e9" containerName="registry-server"
Jul 28 23:29:08.074682 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.074664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.076961 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.076943    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6cltk\""
Jul 28 23:29:08.081762 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.081733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"]
Jul 28 23:29:08.148261 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.148225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67v6q\" (UniqueName: \"kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.148489 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.148414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.148489 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.148464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.248963 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.248926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.249165 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.248973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.249165 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.249007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67v6q\" (UniqueName: \"kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.249423 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.249400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.249517 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.249444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.257207 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.257180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67v6q\" (UniqueName: \"kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.385271 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.385165    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:08.474415 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.474379    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"]
Jul 28 23:29:08.481459 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.481430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.487287 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.487258    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"]
Jul 28 23:29:08.523914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.523888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"]
Jul 28 23:29:08.526312 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:08.526284    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6340f7e_7567_4c9f_8e30_677866e5ab3c.slice/crio-5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0 WatchSource:0}: Error finding container 5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0: Status 404 returned error can't find the container with id 5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0
Jul 28 23:29:08.551901 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.551866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw92z\" (UniqueName: \"kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.552038 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.552018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.552097 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.552082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.653451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.653419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.653586 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.653478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.653586 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.653515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw92z\" (UniqueName: \"kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.653777 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.653759    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.653816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.653785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.660950 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.660925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw92z\" (UniqueName: \"kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.795776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.795740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:08.928271 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:08.928246    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"]
Jul 28 23:29:08.930232 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:08.930199    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f9c0cc_dca2_4907_9ba6_49b9ca8b3448.slice/crio-dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3 WatchSource:0}: Error finding container dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3: Status 404 returned error can't find the container with id dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3
Jul 28 23:29:09.072220 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.072184    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"]
Jul 28 23:29:09.076091 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.076068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.082284 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.082254    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"]
Jul 28 23:29:09.157538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.157505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.157708 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.157584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.157708 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.157610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkxd6\" (UniqueName: \"kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.259054 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.258960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.259054 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.259044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.259282 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.259066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkxd6\" (UniqueName: \"kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.259505 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.259471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.259609 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.259560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.267170 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.267146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkxd6\" (UniqueName: \"kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.312597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.312562    2564 generic.go:356] "Generic (PLEG): container finished" podID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerID="f089bf353b26c3edd8b9e75f82af09561a5d8ed6070b8a0d763d8b17d8b1f77e" exitCode=0
Jul 28 23:29:09.312764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.312655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87" event={"ID":"d6340f7e-7567-4c9f-8e30-677866e5ab3c","Type":"ContainerDied","Data":"f089bf353b26c3edd8b9e75f82af09561a5d8ed6070b8a0d763d8b17d8b1f77e"}
Jul 28 23:29:09.312764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.312714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87" event={"ID":"d6340f7e-7567-4c9f-8e30-677866e5ab3c","Type":"ContainerStarted","Data":"5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0"}
Jul 28 23:29:09.314364 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.314323    2564 generic.go:356] "Generic (PLEG): container finished" podID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerID="fd308bc716f9065d5d3cc223a3567a261d0e07e53eeb5b84fb8f119fecc2b198" exitCode=0
Jul 28 23:29:09.314482 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.314396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr" event={"ID":"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448","Type":"ContainerDied","Data":"fd308bc716f9065d5d3cc223a3567a261d0e07e53eeb5b84fb8f119fecc2b198"}
Jul 28 23:29:09.314482 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.314443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr" event={"ID":"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448","Type":"ContainerStarted","Data":"dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3"}
Jul 28 23:29:09.414706 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.414675    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:09.551247 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.551221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"]
Jul 28 23:29:09.553147 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:09.553106    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b67969_083c_4302_bba9_04e1a0530f23.slice/crio-46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17 WatchSource:0}: Error finding container 46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17: Status 404 returned error can't find the container with id 46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17
Jul 28 23:29:09.670407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.670374    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"]
Jul 28 23:29:09.674109 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.674085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.680942 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.680916    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"]
Jul 28 23:29:09.764669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.764634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.764838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.764676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s65px\" (UniqueName: \"kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.764838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.764702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.865523 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.865434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.865523 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.865481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s65px\" (UniqueName: \"kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.865523 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.865501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.865867 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.865849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.866038 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.866011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:09.873517 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:09.873487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s65px\" (UniqueName: \"kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:10.006820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.006792    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:10.151924 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.151896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"]
Jul 28 23:29:10.153605 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:10.153577    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e60e5d_3582_477a_b064_6787cffa20f1.slice/crio-a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53 WatchSource:0}: Error finding container a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53: Status 404 returned error can't find the container with id a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53
Jul 28 23:29:10.321984 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.321949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerStarted","Data":"5d98f403ac1021d5df8ec843101b0e2e81ac414004574a2e88e599815eb8e6db"}
Jul 28 23:29:10.322159 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.322036    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerStarted","Data":"a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53"}
Jul 28 23:29:10.323565 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.323536    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4b67969-083c-4302-bba9-04e1a0530f23" containerID="cf40c4b446291ae061dc655a51d8bbc103ea041c6207cdeff09b5f8ed1bb6f6e" exitCode=0
Jul 28 23:29:10.323710 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.323623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf" event={"ID":"a4b67969-083c-4302-bba9-04e1a0530f23","Type":"ContainerDied","Data":"cf40c4b446291ae061dc655a51d8bbc103ea041c6207cdeff09b5f8ed1bb6f6e"}
Jul 28 23:29:10.323710 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.323672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf" event={"ID":"a4b67969-083c-4302-bba9-04e1a0530f23","Type":"ContainerStarted","Data":"46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17"}
Jul 28 23:29:10.325720 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.325699    2564 generic.go:356] "Generic (PLEG): container finished" podID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerID="59a49f738e2f4123b424a7d4f360199b0baaf37dbdd645342f1b4ad7c3d7787c" exitCode=0
Jul 28 23:29:10.325809 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:10.325734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr" event={"ID":"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448","Type":"ContainerDied","Data":"59a49f738e2f4123b424a7d4f360199b0baaf37dbdd645342f1b4ad7c3d7787c"}
Jul 28 23:29:11.332171 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.332081    2564 generic.go:356] "Generic (PLEG): container finished" podID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerID="5d98f403ac1021d5df8ec843101b0e2e81ac414004574a2e88e599815eb8e6db" exitCode=0
Jul 28 23:29:11.332621 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.332171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerDied","Data":"5d98f403ac1021d5df8ec843101b0e2e81ac414004574a2e88e599815eb8e6db"}
Jul 28 23:29:11.334110 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.334089    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4b67969-083c-4302-bba9-04e1a0530f23" containerID="14ec75b5616c10a17f49cfa8b1be35ba15b219077080ca3e17a76b49bcf8261a" exitCode=0
Jul 28 23:29:11.334234 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.334174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf" event={"ID":"a4b67969-083c-4302-bba9-04e1a0530f23","Type":"ContainerDied","Data":"14ec75b5616c10a17f49cfa8b1be35ba15b219077080ca3e17a76b49bcf8261a"}
Jul 28 23:29:11.336103 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.336084    2564 generic.go:356] "Generic (PLEG): container finished" podID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerID="d77782c705513f56b8e95888696bf77449ae92d5e3f0ae2f45a5446136385416" exitCode=0
Jul 28 23:29:11.336195 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.336127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87" event={"ID":"d6340f7e-7567-4c9f-8e30-677866e5ab3c","Type":"ContainerDied","Data":"d77782c705513f56b8e95888696bf77449ae92d5e3f0ae2f45a5446136385416"}
Jul 28 23:29:11.338383 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.338361    2564 generic.go:356] "Generic (PLEG): container finished" podID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerID="784243847a3ca39b0c5b12e7223c6c25bce574f7698655fa6e050982d5c233d9" exitCode=0
Jul 28 23:29:11.338463 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:11.338408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr" event={"ID":"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448","Type":"ContainerDied","Data":"784243847a3ca39b0c5b12e7223c6c25bce574f7698655fa6e050982d5c233d9"}
Jul 28 23:29:12.346432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.346397    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4b67969-083c-4302-bba9-04e1a0530f23" containerID="3d0d14da205739431433520e28c8584550ec6897006c6e9047c01d5402251ba2" exitCode=0
Jul 28 23:29:12.346839 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.346474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf" event={"ID":"a4b67969-083c-4302-bba9-04e1a0530f23","Type":"ContainerDied","Data":"3d0d14da205739431433520e28c8584550ec6897006c6e9047c01d5402251ba2"}
Jul 28 23:29:12.348611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.348587    2564 generic.go:356] "Generic (PLEG): container finished" podID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerID="9824e2707b0eb5074d28521ab002e2acfb6d0c9972641175b06796619bc50b43" exitCode=0
Jul 28 23:29:12.348745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.348666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87" event={"ID":"d6340f7e-7567-4c9f-8e30-677866e5ab3c","Type":"ContainerDied","Data":"9824e2707b0eb5074d28521ab002e2acfb6d0c9972641175b06796619bc50b43"}
Jul 28 23:29:12.507124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.507097    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:12.589627 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.589601    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle\") pod \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") "
Jul 28 23:29:12.589738 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.589666    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw92z\" (UniqueName: \"kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z\") pod \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") "
Jul 28 23:29:12.589780 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.589752    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util\") pod \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\" (UID: \"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448\") "
Jul 28 23:29:12.590201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.590175    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle" (OuterVolumeSpecName: "bundle") pod "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" (UID: "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:12.591772 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.591748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z" (OuterVolumeSpecName: "kube-api-access-fw92z") pod "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" (UID: "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448"). InnerVolumeSpecName "kube-api-access-fw92z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:12.594697 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.594663    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util" (OuterVolumeSpecName: "util") pod "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" (UID: "34f9c0cc-dca2-4907-9ba6-49b9ca8b3448"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:12.690617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.690583    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw92z\" (UniqueName: \"kubernetes.io/projected/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-kube-api-access-fw92z\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:12.690617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.690611    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:12.690617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:12.690622    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f9c0cc-dca2-4907-9ba6-49b9ca8b3448-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.356312 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.356273    2564 generic.go:356] "Generic (PLEG): container finished" podID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerID="190d8662b53af9bb924067dbdd2f32a61e85d595884550e5e4b1408a54606d4e" exitCode=0
Jul 28 23:29:13.356791 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.356365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerDied","Data":"190d8662b53af9bb924067dbdd2f32a61e85d595884550e5e4b1408a54606d4e"}
Jul 28 23:29:13.358503 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.358481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr" event={"ID":"34f9c0cc-dca2-4907-9ba6-49b9ca8b3448","Type":"ContainerDied","Data":"dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3"}
Jul 28 23:29:13.358618 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.358519    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr"
Jul 28 23:29:13.358618 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.358530    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dad3b8dff43ed679d42f61cb46091de6c873f6b0bd328ac9c2b70f7f7decb0a3"
Jul 28 23:29:13.577630 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.577606    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:13.581033 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.581008    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:13.701186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701095    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67v6q\" (UniqueName: \"kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q\") pod \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") "
Jul 28 23:29:13.701395 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701199    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle\") pod \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") "
Jul 28 23:29:13.701395 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701304    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util\") pod \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\" (UID: \"d6340f7e-7567-4c9f-8e30-677866e5ab3c\") "
Jul 28 23:29:13.701395 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701339    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkxd6\" (UniqueName: \"kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6\") pod \"a4b67969-083c-4302-bba9-04e1a0530f23\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") "
Jul 28 23:29:13.701535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701402    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle\") pod \"a4b67969-083c-4302-bba9-04e1a0530f23\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") "
Jul 28 23:29:13.701535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701470    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util\") pod \"a4b67969-083c-4302-bba9-04e1a0530f23\" (UID: \"a4b67969-083c-4302-bba9-04e1a0530f23\") "
Jul 28 23:29:13.701871 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.701837    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle" (OuterVolumeSpecName: "bundle") pod "d6340f7e-7567-4c9f-8e30-677866e5ab3c" (UID: "d6340f7e-7567-4c9f-8e30-677866e5ab3c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:13.702053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.702018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle" (OuterVolumeSpecName: "bundle") pod "a4b67969-083c-4302-bba9-04e1a0530f23" (UID: "a4b67969-083c-4302-bba9-04e1a0530f23"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:13.703952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.703927    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q" (OuterVolumeSpecName: "kube-api-access-67v6q") pod "d6340f7e-7567-4c9f-8e30-677866e5ab3c" (UID: "d6340f7e-7567-4c9f-8e30-677866e5ab3c"). InnerVolumeSpecName "kube-api-access-67v6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:13.703952 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.703944    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6" (OuterVolumeSpecName: "kube-api-access-wkxd6") pod "a4b67969-083c-4302-bba9-04e1a0530f23" (UID: "a4b67969-083c-4302-bba9-04e1a0530f23"). InnerVolumeSpecName "kube-api-access-wkxd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:13.707779 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.707749    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util" (OuterVolumeSpecName: "util") pod "a4b67969-083c-4302-bba9-04e1a0530f23" (UID: "a4b67969-083c-4302-bba9-04e1a0530f23"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:13.783536 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.783499    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util" (OuterVolumeSpecName: "util") pod "d6340f7e-7567-4c9f-8e30-677866e5ab3c" (UID: "d6340f7e-7567-4c9f-8e30-677866e5ab3c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:13.802709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802681    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.802709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802706    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67v6q\" (UniqueName: \"kubernetes.io/projected/d6340f7e-7567-4c9f-8e30-677866e5ab3c-kube-api-access-67v6q\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.802709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802716    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.802920 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802724    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6340f7e-7567-4c9f-8e30-677866e5ab3c-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.802920 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802734    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkxd6\" (UniqueName: \"kubernetes.io/projected/a4b67969-083c-4302-bba9-04e1a0530f23-kube-api-access-wkxd6\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:13.802920 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:13.802744    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4b67969-083c-4302-bba9-04e1a0530f23-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:14.366617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.366584    2564 generic.go:356] "Generic (PLEG): container finished" podID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerID="00fb45e04feb0de4eae1b95e930a76084edd41032557ca9bed9c060e048ae3c8" exitCode=0
Jul 28 23:29:14.367055 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.366664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerDied","Data":"00fb45e04feb0de4eae1b95e930a76084edd41032557ca9bed9c060e048ae3c8"}
Jul 28 23:29:14.368609 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.368588    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf"
Jul 28 23:29:14.368726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.368611    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf" event={"ID":"a4b67969-083c-4302-bba9-04e1a0530f23","Type":"ContainerDied","Data":"46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17"}
Jul 28 23:29:14.368726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.368652    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46da1e429124ae33f3369a2fc794cb7359be07ee9e562e5a3fc0499744e02c17"
Jul 28 23:29:14.370726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.370702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87" event={"ID":"d6340f7e-7567-4c9f-8e30-677866e5ab3c","Type":"ContainerDied","Data":"5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0"}
Jul 28 23:29:14.370873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.370751    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b39be24134ac5704a5e7c9e7e8c029a2df505dae8c7bf31eb26d606c48136b0"
Jul 28 23:29:14.370873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:14.370706    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87"
Jul 28 23:29:15.514011 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.513982    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:15.620027 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.619997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util\") pod \"e0e60e5d-3582-477a-b064-6787cffa20f1\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") "
Jul 28 23:29:15.620217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.620093    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s65px\" (UniqueName: \"kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px\") pod \"e0e60e5d-3582-477a-b064-6787cffa20f1\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") "
Jul 28 23:29:15.620217 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.620116    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle\") pod \"e0e60e5d-3582-477a-b064-6787cffa20f1\" (UID: \"e0e60e5d-3582-477a-b064-6787cffa20f1\") "
Jul 28 23:29:15.620620 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.620595    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle" (OuterVolumeSpecName: "bundle") pod "e0e60e5d-3582-477a-b064-6787cffa20f1" (UID: "e0e60e5d-3582-477a-b064-6787cffa20f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:15.622236 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.622205    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px" (OuterVolumeSpecName: "kube-api-access-s65px") pod "e0e60e5d-3582-477a-b064-6787cffa20f1" (UID: "e0e60e5d-3582-477a-b064-6787cffa20f1"). InnerVolumeSpecName "kube-api-access-s65px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:15.625481 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.625438    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util" (OuterVolumeSpecName: "util") pod "e0e60e5d-3582-477a-b064-6787cffa20f1" (UID: "e0e60e5d-3582-477a-b064-6787cffa20f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:15.721609 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.721574    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s65px\" (UniqueName: \"kubernetes.io/projected/e0e60e5d-3582-477a-b064-6787cffa20f1-kube-api-access-s65px\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:15.721609 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.721609    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:15.721798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:15.721619    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0e60e5d-3582-477a-b064-6787cffa20f1-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:16.384083 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:16.383980    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr"
Jul 28 23:29:16.384083 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:16.384013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr" event={"ID":"e0e60e5d-3582-477a-b064-6787cffa20f1","Type":"ContainerDied","Data":"a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53"}
Jul 28 23:29:16.384276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:16.384103    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6437cc6588e7800df982e6c9bfa731ada92640b14c65a0942db72366ea8ea53"
Jul 28 23:29:18.304292 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.304197    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c7d57769f-v5nj4" podUID="862b97c6-82c5-453e-9272-b59e4d7f389a" containerName="console" containerID="cri-o://a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b" gracePeriod=15
Jul 28 23:29:18.558287 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.558226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:29:18.650890 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.650854    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx5kl\" (UniqueName: \"kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651078 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.650910    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651078 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.650938    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651078 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.650978    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651078 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651014    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651283 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651094    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651283 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651130    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca\") pod \"862b97c6-82c5-453e-9272-b59e4d7f389a\" (UID: \"862b97c6-82c5-453e-9272-b59e4d7f389a\") "
Jul 28 23:29:18.651540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:29:18.651612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651562    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config" (OuterVolumeSpecName: "console-config") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:29:18.651612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651565    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:29:18.651724 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651624    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca" (OuterVolumeSpecName: "service-ca") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:29:18.651808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651792    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.651868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651809    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-trusted-ca-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.651868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651818    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-console-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.651868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.651827    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/862b97c6-82c5-453e-9272-b59e4d7f389a-oauth-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.653185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.653158    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl" (OuterVolumeSpecName: "kube-api-access-qx5kl") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "kube-api-access-qx5kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:18.653185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.653158    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:29:18.653339 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.653258    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "862b97c6-82c5-453e-9272-b59e4d7f389a" (UID: "862b97c6-82c5-453e-9272-b59e4d7f389a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:29:18.753125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.753081    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.753125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.753117    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx5kl\" (UniqueName: \"kubernetes.io/projected/862b97c6-82c5-453e-9272-b59e4d7f389a-kube-api-access-qx5kl\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:18.753125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:18.753127    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/862b97c6-82c5-453e-9272-b59e4d7f389a-console-oauth-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:19.401977 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.401936    2564 generic.go:356] "Generic (PLEG): container finished" podID="862b97c6-82c5-453e-9272-b59e4d7f389a" containerID="a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b" exitCode=0
Jul 28 23:29:19.402431 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.401983    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7d57769f-v5nj4"
Jul 28 23:29:19.402431 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.401989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d57769f-v5nj4" event={"ID":"862b97c6-82c5-453e-9272-b59e4d7f389a","Type":"ContainerDied","Data":"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"}
Jul 28 23:29:19.402431 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.402046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7d57769f-v5nj4" event={"ID":"862b97c6-82c5-453e-9272-b59e4d7f389a","Type":"ContainerDied","Data":"a925ececb70289d6ff917e56718df4c55f1aa825e788ad19e5e10556d617b138"}
Jul 28 23:29:19.402431 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.402077    2564 scope.go:117] "RemoveContainer" containerID="a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"
Jul 28 23:29:19.411290 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.411273    2564 scope.go:117] "RemoveContainer" containerID="a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"
Jul 28 23:29:19.411608 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:19.411589    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b\": container with ID starting with a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b not found: ID does not exist" containerID="a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"
Jul 28 23:29:19.411671 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.411615    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b"} err="failed to get container status \"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b\": rpc error: code = NotFound desc = could not find container \"a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b\": container with ID starting with a0c9b32c5913abb6843658750d0425f8e6f17d7dcd3354246bcf8a483e2b926b not found: ID does not exist"
Jul 28 23:29:19.423655 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.423624    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:29:19.427218 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:19.427193    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c7d57769f-v5nj4"]
Jul 28 23:29:20.114934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:20.114901    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="862b97c6-82c5-453e-9272-b59e4d7f389a" path="/var/lib/kubelet/pods/862b97c6-82c5-453e-9272-b59e4d7f389a/volumes"
Jul 28 23:29:21.050802 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:21.050758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:21.638730 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:21.638700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:25.084907 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.084871    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-49px7"]
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="pull"
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085256    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="pull"
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085266    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="pull"
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085271    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="pull"
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085276    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="extract"
Jul 28 23:29:25.085285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085282    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085301    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085307    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085314    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085319    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085329    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085334    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085355    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085361    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085369    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085373    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085382    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="862b97c6-82c5-453e-9272-b59e4d7f389a" containerName="console"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085387    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="862b97c6-82c5-453e-9272-b59e4d7f389a" containerName="console"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085392    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085397    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085405    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085411    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="util"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085416    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="pull"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085420    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="pull"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085426    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="pull"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085430    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="pull"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085489    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b67969-083c-4302-bba9-04e1a0530f23" containerName="extract"
Jul 28 23:29:25.085491 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085497    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e60e5d-3582-477a-b064-6787cffa20f1" containerName="extract"
Jul 28 23:29:25.086146 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085503    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="862b97c6-82c5-453e-9272-b59e4d7f389a" containerName="console"
Jul 28 23:29:25.086146 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085509    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6340f7e-7567-4c9f-8e30-677866e5ab3c" containerName="extract"
Jul 28 23:29:25.086146 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.085517    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="34f9c0cc-dca2-4907-9ba6-49b9ca8b3448" containerName="extract"
Jul 28 23:29:25.089808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.089791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:25.092419 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.092392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vcpqm\""
Jul 28 23:29:25.095972 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.095946    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-49px7"]
Jul 28 23:29:25.213420 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.213378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78brw\" (UniqueName: \"kubernetes.io/projected/f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8-kube-api-access-78brw\") pod \"authorino-operator-75cc8d5cb7-49px7\" (UID: \"f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:25.314953 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.314912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78brw\" (UniqueName: \"kubernetes.io/projected/f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8-kube-api-access-78brw\") pod \"authorino-operator-75cc8d5cb7-49px7\" (UID: \"f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:25.323238 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.323203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78brw\" (UniqueName: \"kubernetes.io/projected/f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8-kube-api-access-78brw\") pod \"authorino-operator-75cc8d5cb7-49px7\" (UID: \"f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:25.401681 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.401590    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:25.538805 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:25.538781    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-49px7"]
Jul 28 23:29:25.541451 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:25.541424    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bd4c9b_9c95_43b5_9209_6da15f4eeeb8.slice/crio-34e645b2c75c2c7903a0ef3bc070f890b085f776bce237410fb1138951028e3f WatchSource:0}: Error finding container 34e645b2c75c2c7903a0ef3bc070f890b085f776bce237410fb1138951028e3f: Status 404 returned error can't find the container with id 34e645b2c75c2c7903a0ef3bc070f890b085f776bce237410fb1138951028e3f
Jul 28 23:29:26.049852 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:26.049819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:26.444875 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:26.444782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7" event={"ID":"f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8","Type":"ContainerStarted","Data":"34e645b2c75c2c7903a0ef3bc070f890b085f776bce237410fb1138951028e3f"}
Jul 28 23:29:28.267794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:28.267705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:28.457330 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:28.457291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7" event={"ID":"f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8","Type":"ContainerStarted","Data":"2f328f809271c4bd98214d4f516cbf313e677de98a19a610b145f3fcd1d1a8b3"}
Jul 28 23:29:28.457540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:28.457473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:28.477352 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:28.477288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7" podStartSLOduration=1.135656483 podStartE2EDuration="3.477273524s" totalImagesPullingTime="2.341617041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:28.474641513 +0000 UTC m=+469.001690776" watchObservedRunningTime="2026-07-28 23:29:28.477273524 +0000 UTC m=+469.004322786"
Jul 28 23:29:29.826425 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.826385    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"]
Jul 28 23:29:29.830113 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.830096    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:29.832615 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.832592    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z6vrk\""
Jul 28 23:29:29.839492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.839469    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"]
Jul 28 23:29:29.964909 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.964875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:29.965062 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:29.965006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrljf\" (UniqueName: \"kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.065950 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.065913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrljf\" (UniqueName: \"kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.066112 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.065960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.066294 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.066278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.074567 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.074543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrljf\" (UniqueName: \"kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.141224 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.141140    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:30.289396 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.289301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"]
Jul 28 23:29:30.290592 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:30.290555    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39da1cb_31e3_47c1_8beb_d7ece3a7ee54.slice/crio-2100f3ad59456d0d3aba6384165a05632462cef3b1d62235b94d591232d62f03 WatchSource:0}: Error finding container 2100f3ad59456d0d3aba6384165a05632462cef3b1d62235b94d591232d62f03: Status 404 returned error can't find the container with id 2100f3ad59456d0d3aba6384165a05632462cef3b1d62235b94d591232d62f03
Jul 28 23:29:30.469267 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.469180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g" event={"ID":"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54","Type":"ContainerStarted","Data":"2100f3ad59456d0d3aba6384165a05632462cef3b1d62235b94d591232d62f03"}
Jul 28 23:29:30.901890 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:30.901850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:31.076240 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:31.076168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:32.671501 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:32.671465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:33.043374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.043320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"]
Jul 28 23:29:33.047545 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.047518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:33.050138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.050112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-chrzc\""
Jul 28 23:29:33.069318 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.069283    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"]
Jul 28 23:29:33.193841 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.193795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf6kn\" (UniqueName: \"kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn\") pod \"limitador-operator-controller-manager-859d7b78bd-vdchq\" (UID: \"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:33.295676 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.295414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf6kn\" (UniqueName: \"kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn\") pod \"limitador-operator-controller-manager-859d7b78bd-vdchq\" (UID: \"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:33.305078 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.305048    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf6kn\" (UniqueName: \"kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn\") pod \"limitador-operator-controller-manager-859d7b78bd-vdchq\" (UID: \"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:33.368976 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.368937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:33.536708 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:33.536678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"]
Jul 28 23:29:34.253520 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:34.253464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:34.659553 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:34.659513    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f11e4a_0e0b_4cae_9c0c_2f518810d8f1.slice/crio-4c2f08e6dad3b2ce97b009829d02572f792907425e1e8cfa96c093df2a4020dd WatchSource:0}: Error finding container 4c2f08e6dad3b2ce97b009829d02572f792907425e1e8cfa96c093df2a4020dd: Status 404 returned error can't find the container with id 4c2f08e6dad3b2ce97b009829d02572f792907425e1e8cfa96c093df2a4020dd
Jul 28 23:29:35.511817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:35.511599    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g" event={"ID":"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54","Type":"ContainerStarted","Data":"c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62"}
Jul 28 23:29:35.511817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:35.511689    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:35.513357 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:35.513311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq" event={"ID":"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1","Type":"ContainerStarted","Data":"4c2f08e6dad3b2ce97b009829d02572f792907425e1e8cfa96c093df2a4020dd"}
Jul 28 23:29:35.532266 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:35.532204    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g" podStartSLOduration=2.107303157 podStartE2EDuration="6.532186334s" totalImagesPullingTime="4.424883177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:35.530009986 +0000 UTC m=+476.057059251" watchObservedRunningTime="2026-07-28 23:29:35.532186334 +0000 UTC m=+476.059235603"
Jul 28 23:29:35.677500 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:35.677461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:36.520638 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:36.520527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq" event={"ID":"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1","Type":"ContainerStarted","Data":"35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4"}
Jul 28 23:29:36.521091 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:36.520729    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:36.536159 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:36.536103    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq" podStartSLOduration=2.057895679 podStartE2EDuration="3.536086018s" totalImagesPullingTime="1.478190339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:36.535063214 +0000 UTC m=+477.062112477" watchObservedRunningTime="2026-07-28 23:29:36.536086018 +0000 UTC m=+477.063135282"
Jul 28 23:29:37.566185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:37.566148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:37.660396 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:37.660334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:37.761075 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:37.761038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:37.861406 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:37.861293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:38.045925 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:38.045891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:38.099166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:38.099128    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:39.465546 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:39.465519    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-49px7"
Jul 28 23:29:39.944047 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:39.944017    2564 scope.go:117] "RemoveContainer" containerID="5ae330fe08b471bc3ae965b818819932a2030a96dacbcf2a0a13b737a31cb383"
Jul 28 23:29:41.648421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:41.648388    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:42.164884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:42.164845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:46.522784 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:46.522746    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:47.529870 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:47.529836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.261873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.261837    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:29:48.267904 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.267873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.281512 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.281482    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:29:48.292740 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.292708    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"]
Jul 28 23:29:48.292911 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.292888    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" containerName="manager" containerID="cri-o://c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62" gracePeriod=2
Jul 28 23:29:48.308940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.308911    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"]
Jul 28 23:29:48.316109 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.316064    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\": pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.325984 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.325635    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"]
Jul 28 23:29:48.326163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.325981    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" containerName="manager" containerID="cri-o://35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4" gracePeriod=2
Jul 28 23:29:48.328192 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.328157    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.330849 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.330809    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.330981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.330949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.331282 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.331258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtj8p\" (UniqueName: \"kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.333081 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.333058    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"]
Jul 28 23:29:48.347439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.347405    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"]
Jul 28 23:29:48.347878 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.347860    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" containerName="manager"
Jul 28 23:29:48.347981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.347881    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" containerName="manager"
Jul 28 23:29:48.347981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.347915    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" containerName="manager"
Jul 28 23:29:48.347981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.347924    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" containerName="manager"
Jul 28 23:29:48.348136 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.348031    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" containerName="manager"
Jul 28 23:29:48.348136 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.348051    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" containerName="manager"
Jul 28 23:29:48.351228 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.351207    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:48.354176 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.354148    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.362161 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.362126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"]
Jul 28 23:29:48.367633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.367597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:48.376195 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.376156    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.432309 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.432274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.432540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.432405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtj8p\" (UniqueName: \"kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.432540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.432441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjj5d\" (UniqueName: \"kubernetes.io/projected/dc572114-b48d-403e-af1d-5f6e42dfdefc-kube-api-access-rjj5d\") pod \"limitador-operator-controller-manager-859d7b78bd-v7469\" (UID: \"dc572114-b48d-403e-af1d-5f6e42dfdefc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:48.432875 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.432847    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.442210 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.442187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtj8p\" (UniqueName: \"kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p\") pod \"kuadrant-operator-controller-manager-86878c769f-wplbt\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.503118 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.503064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:48.532940 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.532905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjj5d\" (UniqueName: \"kubernetes.io/projected/dc572114-b48d-403e-af1d-5f6e42dfdefc-kube-api-access-rjj5d\") pod \"limitador-operator-controller-manager-859d7b78bd-v7469\" (UID: \"dc572114-b48d-403e-af1d-5f6e42dfdefc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:48.550463 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.550426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjj5d\" (UniqueName: \"kubernetes.io/projected/dc572114-b48d-403e-af1d-5f6e42dfdefc-kube-api-access-rjj5d\") pod \"limitador-operator-controller-manager-859d7b78bd-v7469\" (UID: \"dc572114-b48d-403e-af1d-5f6e42dfdefc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:48.577545 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.577519    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.578654 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.578634    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:48.580637 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.580619    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.581492 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.581469    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.584139 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.584115    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.586855 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.586829    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.587044 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.587026    2564 generic.go:356] "Generic (PLEG): container finished" podID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" containerID="c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62" exitCode=0
Jul 28 23:29:48.587115 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.587068    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.587179 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.587121    2564 scope.go:117] "RemoveContainer" containerID="c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62"
Jul 28 23:29:48.588745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.588713    2564 generic.go:356] "Generic (PLEG): container finished" podID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" containerID="35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4" exitCode=0
Jul 28 23:29:48.588871 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.588766    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.589487 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.589461    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.591924 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.591901    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.594142 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.594122    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.596634 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.596615    2564 scope.go:117] "RemoveContainer" containerID="c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62"
Jul 28 23:29:48.596987 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.596959    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62\": container with ID starting with c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62 not found: ID does not exist" containerID="c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62"
Jul 28 23:29:48.597043 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.596995    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62"} err="failed to get container status \"c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62\": rpc error: code = NotFound desc = could not find container \"c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62\": container with ID starting with c0b808eb459418c8872452cc5af7316fb87c88c35193595fd1498abd4f979f62 not found: ID does not exist"
Jul 28 23:29:48.597043 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.597012    2564 scope.go:117] "RemoveContainer" containerID="35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4"
Jul 28 23:29:48.607034 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.607010    2564 scope.go:117] "RemoveContainer" containerID="35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4"
Jul 28 23:29:48.607320 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.607301    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4\": container with ID starting with 35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4 not found: ID does not exist" containerID="35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4"
Jul 28 23:29:48.607444 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.607329    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4"} err="failed to get container status \"35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4\": rpc error: code = NotFound desc = could not find container \"35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4\": container with ID starting with 35eafe0e048c11c3b2c5aaad4fea01b6c5b981177ba2b43d20e02b18361f00f4 not found: ID does not exist"
Jul 28 23:29:48.634265 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.634166    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume\") pod \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") "
Jul 28 23:29:48.634265 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.634225    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf6kn\" (UniqueName: \"kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn\") pod \"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1\" (UID: \"38f11e4a-0e0b-4cae-9c0c-2f518810d8f1\") "
Jul 28 23:29:48.634426 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.634273    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrljf\" (UniqueName: \"kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf\") pod \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\" (UID: \"f39da1cb-31e3-47c1-8beb-d7ece3a7ee54\") "
Jul 28 23:29:48.635079 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.635036    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" (UID: "f39da1cb-31e3-47c1-8beb-d7ece3a7ee54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:48.636935 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.636913    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn" (OuterVolumeSpecName: "kube-api-access-lf6kn") pod "38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" (UID: "38f11e4a-0e0b-4cae-9c0c-2f518810d8f1"). InnerVolumeSpecName "kube-api-access-lf6kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:48.637228 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.637209    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf" (OuterVolumeSpecName: "kube-api-access-jrljf") pod "f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" (UID: "f39da1cb-31e3-47c1-8beb-d7ece3a7ee54"). InnerVolumeSpecName "kube-api-access-jrljf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:29:48.722151 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.722125    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:29:48.723828 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:48.723797    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1dd6e4_bc19_4e53_99c0_c29b07179cb2.slice/crio-ebcab48a1fa1e260c71d4816c418431d712a2129fb4ed74f048b48b19875e5ed WatchSource:0}: Error finding container ebcab48a1fa1e260c71d4816c418431d712a2129fb4ed74f048b48b19875e5ed: Status 404 returned error can't find the container with id ebcab48a1fa1e260c71d4816c418431d712a2129fb4ed74f048b48b19875e5ed
Jul 28 23:29:48.735393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.735337    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-extensions-socket-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:48.735393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.735394    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf6kn\" (UniqueName: \"kubernetes.io/projected/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1-kube-api-access-lf6kn\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:48.735557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.735404    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrljf\" (UniqueName: \"kubernetes.io/projected/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54-kube-api-access-jrljf\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:29:48.762873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.762774    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:48.901418 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.901385    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.903376 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.903331    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.905279 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.905254    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:48.907227 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:48.907199    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:48.909167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:48.909143    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"]
Jul 28 23:29:48.910809 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:29:48.910787    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc572114_b48d_403e_af1d_5f6e42dfdefc.slice/crio-42a31e4026a16da1854b83fd1d0cca7a871ad089a7519707dc601893e59e69cd WatchSource:0}: Error finding container 42a31e4026a16da1854b83fd1d0cca7a871ad089a7519707dc601893e59e69cd: Status 404 returned error can't find the container with id 42a31e4026a16da1854b83fd1d0cca7a871ad089a7519707dc601893e59e69cd
Jul 28 23:29:49.597016 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.596913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469" event={"ID":"dc572114-b48d-403e-af1d-5f6e42dfdefc","Type":"ContainerStarted","Data":"65630b543431987d3f5805d5a62f5268d782842b1e1fe5b5d68750d2cc82ebd3"}
Jul 28 23:29:49.597016 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.596967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469" event={"ID":"dc572114-b48d-403e-af1d-5f6e42dfdefc","Type":"ContainerStarted","Data":"42a31e4026a16da1854b83fd1d0cca7a871ad089a7519707dc601893e59e69cd"}
Jul 28 23:29:49.597016 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.596992    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:29:49.599196 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:49.599161    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:49.599450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.599422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" event={"ID":"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2","Type":"ContainerStarted","Data":"543b1084c1cf24be428865c9c9b29b8f962b5eef0d206293707e3323a264b0f2"}
Jul 28 23:29:49.599553 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.599473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" event={"ID":"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2","Type":"ContainerStarted","Data":"ebcab48a1fa1e260c71d4816c418431d712a2129fb4ed74f048b48b19875e5ed"}
Jul 28 23:29:49.599598 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.599550    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:29:49.617111 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.617063    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469" podStartSLOduration=1.617048289 podStartE2EDuration="1.617048289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:49.615836445 +0000 UTC m=+490.142885709" watchObservedRunningTime="2026-07-28 23:29:49.617048289 +0000 UTC m=+490.144097551"
Jul 28 23:29:49.617691 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:49.617655    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:49.619533 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:49.619506    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:49.621323 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:49.621301    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:49.639662 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:49.639616    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" podStartSLOduration=1.639600905 podStartE2EDuration="1.639600905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:49.639134963 +0000 UTC m=+490.166184227" watchObservedRunningTime="2026-07-28 23:29:49.639600905 +0000 UTC m=+490.166650168"
Jul 28 23:29:50.114941 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:50.114899    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vdchq\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vdchq"
Jul 28 23:29:50.116887 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:50.116861    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f11e4a-0e0b-4cae-9c0c-2f518810d8f1" path="/var/lib/kubelet/pods/38f11e4a-0e0b-4cae-9c0c-2f518810d8f1/volumes"
Jul 28 23:29:50.116982 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:29:50.116936    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xgf4g\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xgf4g"
Jul 28 23:29:50.117337 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:50.117323    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39da1cb-31e3-47c1-8beb-d7ece3a7ee54" path="/var/lib/kubelet/pods/f39da1cb-31e3-47c1-8beb-d7ece3a7ee54/volumes"
Jul 28 23:29:50.778850 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:50.778791    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:50.876620 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:50.876583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:57.215385 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:57.215291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:29:57.313521 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:29:57.313486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:00.502742 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:00.502710    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:00.557776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:00.557740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:00.607770 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:00.607738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7469"
Jul 28 23:30:00.607946 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:00.607822    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:30:08.096389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:08.096334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:08.196236 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:08.196193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:10.144532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:10.144485    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:10.192507 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:10.192472    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:17.105188 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:17.105147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:17.603540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:17.603500    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:19.414627 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:19.414579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:21.880423 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.880384    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:21.884759 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.884738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:21.887485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.887244    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6cltk\""
Jul 28 23:30:21.887485 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.887362    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 23:30:21.892084 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.892049    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:21.981260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:21.981229    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:22.043830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.043796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm2qj\" (UniqueName: \"kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.044049 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.043939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.144795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.144696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.144795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.144754    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm2qj\" (UniqueName: \"kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.145516 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.145494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.155188 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.155159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm2qj\" (UniqueName: \"kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj\") pod \"limitador-limitador-64c94fd64f-x2nwc\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.198134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.198096    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:22.369756 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.369727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:22.371153 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:30:22.371126    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3272a00a_e2a0_483b_a341_55f4ffdeebcb.slice/crio-f44a8c4988fc7933d93bef94de1dbc275f45c5ed968354b4b2437c995551101d WatchSource:0}: Error finding container f44a8c4988fc7933d93bef94de1dbc275f45c5ed968354b4b2437c995551101d: Status 404 returned error can't find the container with id f44a8c4988fc7933d93bef94de1dbc275f45c5ed968354b4b2437c995551101d
Jul 28 23:30:22.660525 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.660467    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:22.665240 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.665209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:22.667631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.667597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:22.667794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.667776    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jsx9f\""
Jul 28 23:30:22.784532 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.784496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" event={"ID":"3272a00a-e2a0-483b-a341-55f4ffdeebcb","Type":"ContainerStarted","Data":"f44a8c4988fc7933d93bef94de1dbc275f45c5ed968354b4b2437c995551101d"}
Jul 28 23:30:22.852189 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.852153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbbnp\" (UniqueName: \"kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp\") pod \"authorino-58cd955675-f6mtb\" (UID: \"21a3e97f-0992-4b06-9008-886d37ce8a18\") " pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:22.953760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.953670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbbnp\" (UniqueName: \"kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp\") pod \"authorino-58cd955675-f6mtb\" (UID: \"21a3e97f-0992-4b06-9008-886d37ce8a18\") " pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:22.961465 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.961438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbbnp\" (UniqueName: \"kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp\") pod \"authorino-58cd955675-f6mtb\" (UID: \"21a3e97f-0992-4b06-9008-886d37ce8a18\") " pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:22.977136 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:22.977104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:23.112584 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:23.112554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:23.114460 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:30:23.114428    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21a3e97f_0992_4b06_9008_886d37ce8a18.slice/crio-30fcceaedfdf25bcda708925fc9f0073c056d1be687389e095f70e0951c84a28 WatchSource:0}: Error finding container 30fcceaedfdf25bcda708925fc9f0073c056d1be687389e095f70e0951c84a28: Status 404 returned error can't find the container with id 30fcceaedfdf25bcda708925fc9f0073c056d1be687389e095f70e0951c84a28
Jul 28 23:30:23.363571 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:23.363445    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:23.458599 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:23.458518    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:23.796322 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:23.796290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f6mtb" event={"ID":"21a3e97f-0992-4b06-9008-886d37ce8a18","Type":"ContainerStarted","Data":"30fcceaedfdf25bcda708925fc9f0073c056d1be687389e095f70e0951c84a28"}
Jul 28 23:30:25.816689 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:25.816649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" event={"ID":"3272a00a-e2a0-483b-a341-55f4ffdeebcb","Type":"ContainerStarted","Data":"9e8bf40920b5356638d15863019857249e9218f2784471daf7fa43f1c29d6103"}
Jul 28 23:30:25.817237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:25.817208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:25.834276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:25.834219    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" podStartSLOduration=1.866907656 podStartE2EDuration="4.834204717s" totalImagesPullingTime="2.967297061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:30:25.831485273 +0000 UTC m=+526.358534537" watchObservedRunningTime="2026-07-28 23:30:25.834204717 +0000 UTC m=+526.361253980"
Jul 28 23:30:29.843513 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:29.843469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f6mtb" event={"ID":"21a3e97f-0992-4b06-9008-886d37ce8a18","Type":"ContainerStarted","Data":"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"}
Jul 28 23:30:29.858403 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:29.858333    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-f6mtb" podStartSLOduration=2.033269884 podStartE2EDuration="7.858317393s" totalImagesPullingTime="5.825047509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:30:29.856181597 +0000 UTC m=+530.383230859" watchObservedRunningTime="2026-07-28 23:30:29.858317393 +0000 UTC m=+530.385366657"
Jul 28 23:30:31.430761 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:31.430726    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:31.855053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:31.855009    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-f6mtb" podUID="21a3e97f-0992-4b06-9008-886d37ce8a18" containerName="authorino" containerID="cri-o://1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7" gracePeriod=30
Jul 28 23:30:32.102169 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.102138    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:32.134781 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.134709    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbbnp\" (UniqueName: \"kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp\") pod \"21a3e97f-0992-4b06-9008-886d37ce8a18\" (UID: \"21a3e97f-0992-4b06-9008-886d37ce8a18\") "
Jul 28 23:30:32.136698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.136672    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp" (OuterVolumeSpecName: "kube-api-access-vbbnp") pod "21a3e97f-0992-4b06-9008-886d37ce8a18" (UID: "21a3e97f-0992-4b06-9008-886d37ce8a18"). InnerVolumeSpecName "kube-api-access-vbbnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:30:32.235496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.235419    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbbnp\" (UniqueName: \"kubernetes.io/projected/21a3e97f-0992-4b06-9008-886d37ce8a18-kube-api-access-vbbnp\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:30:32.861621 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.861589    2564 generic.go:356] "Generic (PLEG): container finished" podID="21a3e97f-0992-4b06-9008-886d37ce8a18" containerID="1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7" exitCode=0
Jul 28 23:30:32.862108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.861634    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f6mtb"
Jul 28 23:30:32.862108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.861690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f6mtb" event={"ID":"21a3e97f-0992-4b06-9008-886d37ce8a18","Type":"ContainerDied","Data":"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"}
Jul 28 23:30:32.862108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.861735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f6mtb" event={"ID":"21a3e97f-0992-4b06-9008-886d37ce8a18","Type":"ContainerDied","Data":"30fcceaedfdf25bcda708925fc9f0073c056d1be687389e095f70e0951c84a28"}
Jul 28 23:30:32.862108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.861752    2564 scope.go:117] "RemoveContainer" containerID="1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"
Jul 28 23:30:32.870833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.870817    2564 scope.go:117] "RemoveContainer" containerID="1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"
Jul 28 23:30:32.871081 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:30:32.871062    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7\": container with ID starting with 1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7 not found: ID does not exist" containerID="1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"
Jul 28 23:30:32.871122 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.871089    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7"} err="failed to get container status \"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7\": rpc error: code = NotFound desc = could not find container \"1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7\": container with ID starting with 1c36bbdddde66a6039c0e207162852f72e1a0c2f9d71b2117ba5ac7ca1223ff7 not found: ID does not exist"
Jul 28 23:30:32.882003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.881978    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:32.887587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:32.887565    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-f6mtb"]
Jul 28 23:30:34.115917 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:34.115885    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21a3e97f-0992-4b06-9008-886d37ce8a18" path="/var/lib/kubelet/pods/21a3e97f-0992-4b06-9008-886d37ce8a18/volumes"
Jul 28 23:30:36.403460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:36.403416    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:36.403912 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:36.403673    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" podUID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" containerName="limitador" containerID="cri-o://9e8bf40920b5356638d15863019857249e9218f2784471daf7fa43f1c29d6103" gracePeriod=30
Jul 28 23:30:36.404464 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:36.404434    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:36.886104 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:36.886072    2564 generic.go:356] "Generic (PLEG): container finished" podID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" containerID="9e8bf40920b5356638d15863019857249e9218f2784471daf7fa43f1c29d6103" exitCode=0
Jul 28 23:30:36.886268 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:36.886143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" event={"ID":"3272a00a-e2a0-483b-a341-55f4ffdeebcb","Type":"ContainerDied","Data":"9e8bf40920b5356638d15863019857249e9218f2784471daf7fa43f1c29d6103"}
Jul 28 23:30:37.365654 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.365630    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:37.381002 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.380965    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm2qj\" (UniqueName: \"kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj\") pod \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") "
Jul 28 23:30:37.381162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.381090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file\") pod \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\" (UID: \"3272a00a-e2a0-483b-a341-55f4ffdeebcb\") "
Jul 28 23:30:37.381537 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.381507    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file" (OuterVolumeSpecName: "config-file") pod "3272a00a-e2a0-483b-a341-55f4ffdeebcb" (UID: "3272a00a-e2a0-483b-a341-55f4ffdeebcb"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:30:37.383252 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.383226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj" (OuterVolumeSpecName: "kube-api-access-fm2qj") pod "3272a00a-e2a0-483b-a341-55f4ffdeebcb" (UID: "3272a00a-e2a0-483b-a341-55f4ffdeebcb"). InnerVolumeSpecName "kube-api-access-fm2qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:30:37.482134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.482046    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3272a00a-e2a0-483b-a341-55f4ffdeebcb-config-file\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:30:37.482134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.482076    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm2qj\" (UniqueName: \"kubernetes.io/projected/3272a00a-e2a0-483b-a341-55f4ffdeebcb-kube-api-access-fm2qj\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:30:37.892631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.892602    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"
Jul 28 23:30:37.892803 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.892650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2nwc" event={"ID":"3272a00a-e2a0-483b-a341-55f4ffdeebcb","Type":"ContainerDied","Data":"f44a8c4988fc7933d93bef94de1dbc275f45c5ed968354b4b2437c995551101d"}
Jul 28 23:30:37.892803 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.892736    2564 scope.go:117] "RemoveContainer" containerID="9e8bf40920b5356638d15863019857249e9218f2784471daf7fa43f1c29d6103"
Jul 28 23:30:37.913274 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.913247    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:37.916421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:37.916399    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2nwc"]
Jul 28 23:30:38.116124 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.116092    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" path="/var/lib/kubelet/pods/3272a00a-e2a0-483b-a341-55f4ffdeebcb/volumes"
Jul 28 23:30:38.705464 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.705425    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"]
Jul 28 23:30:38.706096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706072    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a3e97f-0992-4b06-9008-886d37ce8a18" containerName="authorino"
Jul 28 23:30:38.706096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706099    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a3e97f-0992-4b06-9008-886d37ce8a18" containerName="authorino"
Jul 28 23:30:38.706242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706115    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" containerName="limitador"
Jul 28 23:30:38.706242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706124    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" containerName="limitador"
Jul 28 23:30:38.706242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706226    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3272a00a-e2a0-483b-a341-55f4ffdeebcb" containerName="limitador"
Jul 28 23:30:38.706242 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.706236    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a3e97f-0992-4b06-9008-886d37ce8a18" containerName="authorino"
Jul 28 23:30:38.709409 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.709389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.711859 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.711836    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 23:30:38.712091 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.712035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 23:30:38.712215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.712064    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-7kbrn\""
Jul 28 23:30:38.713467 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.713382    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 23:30:38.716257 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.716237    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"]
Jul 28 23:30:38.792764 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.792732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/faee699a-6256-41e8-8c52-84964b027f3f-data\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.792946 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.792803    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8btfr\" (UniqueName: \"kubernetes.io/projected/faee699a-6256-41e8-8c52-84964b027f3f-kube-api-access-8btfr\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.894030 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.893992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8btfr\" (UniqueName: \"kubernetes.io/projected/faee699a-6256-41e8-8c52-84964b027f3f-kube-api-access-8btfr\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.894184 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.894113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/faee699a-6256-41e8-8c52-84964b027f3f-data\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.894507 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.894492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/faee699a-6256-41e8-8c52-84964b027f3f-data\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:38.901951 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:38.901927    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8btfr\" (UniqueName: \"kubernetes.io/projected/faee699a-6256-41e8-8c52-84964b027f3f-kube-api-access-8btfr\") pod \"postgres-69fb4594cf-d4fl9\" (UID: \"faee699a-6256-41e8-8c52-84964b027f3f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:39.023646 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:39.023556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:39.160943 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:39.160915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"]
Jul 28 23:30:39.162668 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:30:39.162640    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaee699a_6256_41e8_8c52_84964b027f3f.slice/crio-b891e3c924dc650de843be72efe6744df62b0f7a631ee56d4faa9e6065b4f41e WatchSource:0}: Error finding container b891e3c924dc650de843be72efe6744df62b0f7a631ee56d4faa9e6065b4f41e: Status 404 returned error can't find the container with id b891e3c924dc650de843be72efe6744df62b0f7a631ee56d4faa9e6065b4f41e
Jul 28 23:30:39.907795 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:39.907760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9" event={"ID":"faee699a-6256-41e8-8c52-84964b027f3f","Type":"ContainerStarted","Data":"b891e3c924dc650de843be72efe6744df62b0f7a631ee56d4faa9e6065b4f41e"}
Jul 28 23:30:40.441281 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:40.441238    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:40.492092 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:40.492056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:44.121064 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:44.121041    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 23:30:44.941529 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:44.941490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9" event={"ID":"faee699a-6256-41e8-8c52-84964b027f3f","Type":"ContainerStarted","Data":"fc73aeb2b71d0c0c0506085a0ba193259c2a5a918cffd47438075dd715928cc0"}
Jul 28 23:30:44.941700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:44.941559    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:44.956669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:44.956602    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9" podStartSLOduration=2.001881104 podStartE2EDuration="6.956581072s" totalImagesPullingTime="4.954699968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:30:44.95586453 +0000 UTC m=+545.482913793" watchObservedRunningTime="2026-07-28 23:30:44.956581072 +0000 UTC m=+545.483630348"
Jul 28 23:30:50.977219 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:50.977187    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-d4fl9"
Jul 28 23:30:51.283669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.283581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:51.383805 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.383764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:51.794457 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.794421    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:30:51.806170 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.806136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:51.806474 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.806446    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:30:51.809033 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.809002    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jsx9f\""
Jul 28 23:30:51.809182 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.809070    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 23:30:51.917752 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.917711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877mc\" (UniqueName: \"kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:51.917906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:51.917772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.018540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.018504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.018921 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.018596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877mc\" (UniqueName: \"kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.020976 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.020951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.026010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.025987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-877mc\" (UniqueName: \"kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc\") pod \"authorino-9c9944678-nkfrh\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") " pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.116690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.116618    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:30:52.255821 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.255798    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:30:52.257905 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:30:52.257869    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1241c0e_94db_48d1_abb4_cf47eed8631e.slice/crio-ae6b9b11e5e9cb558279f2e5beebd27e29803385dd20cdfe4e1070fcbbc854b5 WatchSource:0}: Error finding container ae6b9b11e5e9cb558279f2e5beebd27e29803385dd20cdfe4e1070fcbbc854b5: Status 404 returned error can't find the container with id ae6b9b11e5e9cb558279f2e5beebd27e29803385dd20cdfe4e1070fcbbc854b5
Jul 28 23:30:52.989013 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.988919    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9c9944678-nkfrh" event={"ID":"d1241c0e-94db-48d1-abb4-cf47eed8631e","Type":"ContainerStarted","Data":"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"}
Jul 28 23:30:52.989013 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:52.988965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9c9944678-nkfrh" event={"ID":"d1241c0e-94db-48d1-abb4-cf47eed8631e","Type":"ContainerStarted","Data":"ae6b9b11e5e9cb558279f2e5beebd27e29803385dd20cdfe4e1070fcbbc854b5"}
Jul 28 23:30:53.004918 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:53.004865    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9c9944678-nkfrh" podStartSLOduration=1.638542693 podStartE2EDuration="2.004848305s" totalImagesPullingTime="366.305612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:30:53.00429387 +0000 UTC m=+553.531343113" watchObservedRunningTime="2026-07-28 23:30:53.004848305 +0000 UTC m=+553.531897572"
Jul 28 23:30:58.217072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:58.217035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:58.308201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:58.308169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:30:58.578131 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:30:58.578101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:01.889445 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:01.889405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:01.987915 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:01.987875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:33.497845 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:33.497807    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:33.597874 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:33.597843    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:34.102722 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.102691    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:34.585849 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.585823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:34.684727 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.684696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:34.712174 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.712137    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"]
Jul 28 23:31:34.716142 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.716118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.718674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.718654    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 23:31:34.718674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.718666    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6rfl2\""
Jul 28 23:31:34.726572 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.726549    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wrxds"]
Jul 28 23:31:34.736554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.736525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"]
Jul 28 23:31:34.736702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.736626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.747223 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.747194    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wrxds"]
Jul 28 23:31:34.774929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.774903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:34.790551 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.790520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11154eb5-663f-4585-ab9c-723b1db23531-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.790702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.790571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.790702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.790625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbx29\" (UniqueName: \"kubernetes.io/projected/11154eb5-663f-4585-ab9c-723b1db23531-kube-api-access-cbx29\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.790702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.790669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7tpm\" (UniqueName: \"kubernetes.io/projected/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-kube-api-access-n7tpm\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.891802 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.891704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7tpm\" (UniqueName: \"kubernetes.io/projected/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-kube-api-access-n7tpm\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.891802 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.891769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11154eb5-663f-4585-ab9c-723b1db23531-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.891802 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.891795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.892071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.891816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbx29\" (UniqueName: \"kubernetes.io/projected/11154eb5-663f-4585-ab9c-723b1db23531-kube-api-access-cbx29\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.892563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.892526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11154eb5-663f-4585-ab9c-723b1db23531-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:34.892563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.892538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.899372 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.899311    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7tpm\" (UniqueName: \"kubernetes.io/projected/3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e-kube-api-access-n7tpm\") pod \"payload-processing-5cc6cd444b-wrxds\" (UID: \"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:34.899631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:34.899608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbx29\" (UniqueName: \"kubernetes.io/projected/11154eb5-663f-4585-ab9c-723b1db23531-kube-api-access-cbx29\") pod \"payload-pre-processing-687dbb6cf6-xmsls\" (UID: \"11154eb5-663f-4585-ab9c-723b1db23531\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:35.028933 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.028899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:35.048202 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.048176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:35.173913 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.173883    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"]
Jul 28 23:31:35.175807 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:31:35.175775    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11154eb5_663f_4585_ab9c_723b1db23531.slice/crio-740213eeb38e83670ec2fa97e274164055cac39d81637b3e9387292f88e26154 WatchSource:0}: Error finding container 740213eeb38e83670ec2fa97e274164055cac39d81637b3e9387292f88e26154: Status 404 returned error can't find the container with id 740213eeb38e83670ec2fa97e274164055cac39d81637b3e9387292f88e26154
Jul 28 23:31:35.204586 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.204560    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wrxds"]
Jul 28 23:31:35.206050 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:31:35.206025    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa31e84_a9ac_4b9e_b2f4_2f6c3ce9539e.slice/crio-8db0aaeae45cc44d9313ac8327c8f28da1dd0db0dcf6c38228ae34154e463c3e WatchSource:0}: Error finding container 8db0aaeae45cc44d9313ac8327c8f28da1dd0db0dcf6c38228ae34154e463c3e: Status 404 returned error can't find the container with id 8db0aaeae45cc44d9313ac8327c8f28da1dd0db0dcf6c38228ae34154e463c3e
Jul 28 23:31:35.223777 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.223745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls" event={"ID":"11154eb5-663f-4585-ab9c-723b1db23531","Type":"ContainerStarted","Data":"740213eeb38e83670ec2fa97e274164055cac39d81637b3e9387292f88e26154"}
Jul 28 23:31:35.225000 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:35.224974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds" event={"ID":"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e","Type":"ContainerStarted","Data":"8db0aaeae45cc44d9313ac8327c8f28da1dd0db0dcf6c38228ae34154e463c3e"}
Jul 28 23:31:37.241724 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.241682    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds" event={"ID":"3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e","Type":"ContainerStarted","Data":"7d4d41a90f00533b6792ebd057f438cfc4ac58bb823c34003c924993d0a4ca87"}
Jul 28 23:31:37.242166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.241743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:37.243235 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.243197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls" event={"ID":"11154eb5-663f-4585-ab9c-723b1db23531","Type":"ContainerStarted","Data":"92027dab773a1b634073c5545d6fca75e7890a4b71caf3459e45f2037cf55014"}
Jul 28 23:31:37.243397 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.243247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:37.259456 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.259339    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds" podStartSLOduration=1.4399123409999999 podStartE2EDuration="3.259324612s" totalImagesPullingTime="1.819412271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:37.256200529 +0000 UTC m=+597.783249794" watchObservedRunningTime="2026-07-28 23:31:37.259324612 +0000 UTC m=+597.786373853"
Jul 28 23:31:37.269963 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:37.269873    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls" podStartSLOduration=1.426317787 podStartE2EDuration="3.269856982s" totalImagesPullingTime="1.843539195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:37.269379655 +0000 UTC m=+597.796428913" watchObservedRunningTime="2026-07-28 23:31:37.269856982 +0000 UTC m=+597.796906247"
Jul 28 23:31:39.952216 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:39.952190    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:31:39.952692 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:39.952387    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:31:39.958663 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:39.958638    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:31:39.958934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:39.958908    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:31:43.148307 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:43.148271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:43.248791 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:43.248752    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:45.159368 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:45.159256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:45.225054 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:45.224996    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:31:48.251258 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:48.251228    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wrxds"
Jul 28 23:31:48.251776 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:48.251294    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xmsls"
Jul 28 23:31:53.933316 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:31:53.933274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:32:12.314058 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.314021    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"]
Jul 28 23:32:12.317792 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.317773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.320596 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.320574    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dlh7r\""
Jul 28 23:32:12.320596 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.320591    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 23:32:12.320766 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.320593    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 23:32:12.321592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.321570    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 23:32:12.326974 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.326952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"]
Jul 28 23:32:12.337552 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.337528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:32:12.438280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.438280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.438553 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.438553 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.438553 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbh2d\" (UniqueName: \"kubernetes.io/projected/0367daa4-4783-4206-97e0-c4c01be7c89b-kube-api-access-dbh2d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.438687 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.438569    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0367daa4-4783-4206-97e0-c4c01be7c89b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539194 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbh2d\" (UniqueName: \"kubernetes.io/projected/0367daa4-4783-4206-97e0-c4c01be7c89b-kube-api-access-dbh2d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0367daa4-4783-4206-97e0-c4c01be7c89b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539711 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.539840 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.539819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.541640 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.541619    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0367daa4-4783-4206-97e0-c4c01be7c89b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.541758 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.541694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0367daa4-4783-4206-97e0-c4c01be7c89b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.546451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.546428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbh2d\" (UniqueName: \"kubernetes.io/projected/0367daa4-4783-4206-97e0-c4c01be7c89b-kube-api-access-dbh2d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8d7mp\" (UID: \"0367daa4-4783-4206-97e0-c4c01be7c89b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.630099 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.630014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:12.767106 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.767071    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"]
Jul 28 23:32:12.768564 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:32:12.768516    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0367daa4_4783_4206_97e0_c4c01be7c89b.slice/crio-be92c175dc928fad80d9cc28d2bdaa584cf94e4aa81fc0111d841f97b8093401 WatchSource:0}: Error finding container be92c175dc928fad80d9cc28d2bdaa584cf94e4aa81fc0111d841f97b8093401: Status 404 returned error can't find the container with id be92c175dc928fad80d9cc28d2bdaa584cf94e4aa81fc0111d841f97b8093401
Jul 28 23:32:12.770192 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:12.770172    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:32:13.463382 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:13.455826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp" event={"ID":"0367daa4-4783-4206-97e0-c4c01be7c89b","Type":"ContainerStarted","Data":"be92c175dc928fad80d9cc28d2bdaa584cf94e4aa81fc0111d841f97b8093401"}
Jul 28 23:32:14.464196 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:14.464156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp" event={"ID":"0367daa4-4783-4206-97e0-c4c01be7c89b","Type":"ContainerStarted","Data":"d2a21436404a065f5faa48435b0dfdc86b0a1b73de6c6f1dd1cf66d73da4a006"}
Jul 28 23:32:14.483818 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:14.483766    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp" podStartSLOduration=1.435419383 podStartE2EDuration="2.483752194s" totalImagesPullingTime="1.048332811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:32:14.480649334 +0000 UTC m=+635.007698635" watchObservedRunningTime="2026-07-28 23:32:14.483752194 +0000 UTC m=+635.010801455"
Jul 28 23:32:21.137495 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:21.137464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:32:21.188805 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:21.188777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:32:22.631163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:22.631123    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:22.631731 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:22.631191    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:22.644030 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:22.644003    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:23.525243 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:23.525207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8d7mp"
Jul 28 23:32:26.129933 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:26.129899    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:32:50.822261 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:32:50.822222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:33:11.647454 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:33:11.647419    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:33:23.427981 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:33:23.427944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:33:43.845108 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:33:43.845022    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:24.803662 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:24.803619    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bd7dd74bc-lj5j5"]
Jul 28 23:34:24.807608 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:24.807586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:24.813660 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:24.813628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bd7dd74bc-lj5j5"]
Jul 28 23:34:24.900932 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:24.900898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4w2g\" (UniqueName: \"kubernetes.io/projected/f910a034-218b-48eb-97da-5308277a0ec3-kube-api-access-h4w2g\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:24.901107 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:24.900947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f910a034-218b-48eb-97da-5308277a0ec3-tls-cert\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.002003 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.001960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f910a034-218b-48eb-97da-5308277a0ec3-tls-cert\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.002188 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.002073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4w2g\" (UniqueName: \"kubernetes.io/projected/f910a034-218b-48eb-97da-5308277a0ec3-kube-api-access-h4w2g\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.004466 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.004439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f910a034-218b-48eb-97da-5308277a0ec3-tls-cert\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.009482 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.009456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4w2g\" (UniqueName: \"kubernetes.io/projected/f910a034-218b-48eb-97da-5308277a0ec3-kube-api-access-h4w2g\") pod \"authorino-6bd7dd74bc-lj5j5\" (UID: \"f910a034-218b-48eb-97da-5308277a0ec3\") " pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.117911 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.117815    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5"
Jul 28 23:34:25.263601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:25.263571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bd7dd74bc-lj5j5"]
Jul 28 23:34:25.265936 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:34:25.265908    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf910a034_218b_48eb_97da_5308277a0ec3.slice/crio-42c2526426f8880a7455a029227dec7f0feae751aaba763c4554cd0f67aac9fe WatchSource:0}: Error finding container 42c2526426f8880a7455a029227dec7f0feae751aaba763c4554cd0f67aac9fe: Status 404 returned error can't find the container with id 42c2526426f8880a7455a029227dec7f0feae751aaba763c4554cd0f67aac9fe
Jul 28 23:34:26.231061 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.231022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5" event={"ID":"f910a034-218b-48eb-97da-5308277a0ec3","Type":"ContainerStarted","Data":"d5d038e4f2274c6d486c9ab42e6cf8a39218c8d722598f348331e025cb1581df"}
Jul 28 23:34:26.231608 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.231074    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5" event={"ID":"f910a034-218b-48eb-97da-5308277a0ec3","Type":"ContainerStarted","Data":"42c2526426f8880a7455a029227dec7f0feae751aaba763c4554cd0f67aac9fe"}
Jul 28 23:34:26.250777 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.250708    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bd7dd74bc-lj5j5" podStartSLOduration=1.781448272 podStartE2EDuration="2.250685677s" totalImagesPullingTime="469.237405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:34:26.248870127 +0000 UTC m=+766.775919392" watchObservedRunningTime="2026-07-28 23:34:26.250685677 +0000 UTC m=+766.777734940"
Jul 28 23:34:26.278145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.278108    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:34:26.278399 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.278328    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-9c9944678-nkfrh" podUID="d1241c0e-94db-48d1-abb4-cf47eed8631e" containerName="authorino" containerID="cri-o://24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085" gracePeriod=30
Jul 28 23:34:26.548273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.548242    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:34:26.618591 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.618557    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-877mc\" (UniqueName: \"kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc\") pod \"d1241c0e-94db-48d1-abb4-cf47eed8631e\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") "
Jul 28 23:34:26.618790 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.618711    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert\") pod \"d1241c0e-94db-48d1-abb4-cf47eed8631e\" (UID: \"d1241c0e-94db-48d1-abb4-cf47eed8631e\") "
Jul 28 23:34:26.621118 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.621087    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc" (OuterVolumeSpecName: "kube-api-access-877mc") pod "d1241c0e-94db-48d1-abb4-cf47eed8631e" (UID: "d1241c0e-94db-48d1-abb4-cf47eed8631e"). InnerVolumeSpecName "kube-api-access-877mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:34:26.633478 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.633444    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d1241c0e-94db-48d1-abb4-cf47eed8631e" (UID: "d1241c0e-94db-48d1-abb4-cf47eed8631e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:34:26.719721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.719689    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1241c0e-94db-48d1-abb4-cf47eed8631e-tls-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:34:26.719721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.719718    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-877mc\" (UniqueName: \"kubernetes.io/projected/d1241c0e-94db-48d1-abb4-cf47eed8631e-kube-api-access-877mc\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:34:26.797413 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.797377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:26.898178 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:26.898140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:27.238163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.238067    2564 generic.go:356] "Generic (PLEG): container finished" podID="d1241c0e-94db-48d1-abb4-cf47eed8631e" containerID="24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085" exitCode=0
Jul 28 23:34:27.238163 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.238125    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9c9944678-nkfrh"
Jul 28 23:34:27.238702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.238161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9c9944678-nkfrh" event={"ID":"d1241c0e-94db-48d1-abb4-cf47eed8631e","Type":"ContainerDied","Data":"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"}
Jul 28 23:34:27.238702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.238206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9c9944678-nkfrh" event={"ID":"d1241c0e-94db-48d1-abb4-cf47eed8631e","Type":"ContainerDied","Data":"ae6b9b11e5e9cb558279f2e5beebd27e29803385dd20cdfe4e1070fcbbc854b5"}
Jul 28 23:34:27.238702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.238229    2564 scope.go:117] "RemoveContainer" containerID="24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"
Jul 28 23:34:27.247805 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.247783    2564 scope.go:117] "RemoveContainer" containerID="24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"
Jul 28 23:34:27.248146 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:34:27.248122    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085\": container with ID starting with 24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085 not found: ID does not exist" containerID="24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"
Jul 28 23:34:27.248219 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.248160    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085"} err="failed to get container status \"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085\": rpc error: code = NotFound desc = could not find container \"24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085\": container with ID starting with 24ce49c031c789dfee371247eb0eb379c1d8ac9eeb34110c5ffa0faa3c668085 not found: ID does not exist"
Jul 28 23:34:27.267171 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.267140    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:34:27.271117 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.271090    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9c9944678-nkfrh"]
Jul 28 23:34:27.892704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.892669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:27.997337 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:27.997305    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:28.117211 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:28.117174    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1241c0e-94db-48d1-abb4-cf47eed8631e" path="/var/lib/kubelet/pods/d1241c0e-94db-48d1-abb4-cf47eed8631e/volumes"
Jul 28 23:34:33.005935 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:33.005900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:33.107587 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:33.107553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:34:57.752954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:34:57.752914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:35:29.004910 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:35:29.004872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:36:31.833317 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:31.833273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:36:33.058201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:33.058162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:36:39.991451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:39.991421    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:36:39.992215 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:39.992190    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:36:39.997913 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:39.997893    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:36:39.998276 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:36:39.998259    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:37:07.280602 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:07.280569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:07.379970 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:07.379937    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:07.492255 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:07.492210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:13.063143 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:13.063105    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:13.163164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:13.163124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:13.262975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:13.262928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:13.387847 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:13.387762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:19.021484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:19.021431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:37:21.038298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:37:21.038267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:40:52.317694 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:40:52.317661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:41:40.038389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:40.038356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:41:40.039550 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:40.039522    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:41:40.046475 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:40.046417    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:41:40.047662 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:40.047636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:41:49.283673 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:49.283633    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:41:49.284100 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:49.283868    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" podUID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" containerName="manager" containerID="cri-o://543b1084c1cf24be428865c9c9b29b8f962b5eef0d206293707e3323a264b0f2" gracePeriod=10
Jul 28 23:41:52.849302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:52.849217    2564 generic.go:356] "Generic (PLEG): container finished" podID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" containerID="543b1084c1cf24be428865c9c9b29b8f962b5eef0d206293707e3323a264b0f2" exitCode=0
Jul 28 23:41:52.849302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:52.849275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" event={"ID":"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2","Type":"ContainerDied","Data":"543b1084c1cf24be428865c9c9b29b8f962b5eef0d206293707e3323a264b0f2"}
Jul 28 23:41:53.241261 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.241225    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:41:53.336185 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.336142    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume\") pod \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") "
Jul 28 23:41:53.336397 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.336205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtj8p\" (UniqueName: \"kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p\") pod \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\" (UID: \"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2\") "
Jul 28 23:41:53.336597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.336569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" (UID: "4a1dd6e4-bc19-4e53-99c0-c29b07179cb2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:41:53.338285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.338263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p" (OuterVolumeSpecName: "kube-api-access-wtj8p") pod "4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" (UID: "4a1dd6e4-bc19-4e53-99c0-c29b07179cb2"). InnerVolumeSpecName "kube-api-access-wtj8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:41:53.437492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.437385    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-extensions-socket-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:41:53.437492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.437422    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtj8p\" (UniqueName: \"kubernetes.io/projected/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2-kube-api-access-wtj8p\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:41:53.856895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.856861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt" event={"ID":"4a1dd6e4-bc19-4e53-99c0-c29b07179cb2","Type":"ContainerDied","Data":"ebcab48a1fa1e260c71d4816c418431d712a2129fb4ed74f048b48b19875e5ed"}
Jul 28 23:41:53.857311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.856924    2564 scope.go:117] "RemoveContainer" containerID="543b1084c1cf24be428865c9c9b29b8f962b5eef0d206293707e3323a264b0f2"
Jul 28 23:41:53.857311 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.856872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"
Jul 28 23:41:53.878313 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.878285    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:41:53.885157 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:53.885133    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplbt"]
Jul 28 23:41:54.116720 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:41:54.116637    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" path="/var/lib/kubelet/pods/4a1dd6e4-bc19-4e53-99c0-c29b07179cb2/volumes"
Jul 28 23:42:33.054677 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:33.054588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:42:50.402260 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402228    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"]
Jul 28 23:42:50.402782 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402764    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" containerName="manager"
Jul 28 23:42:50.402830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402784    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" containerName="manager"
Jul 28 23:42:50.402830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402794    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1241c0e-94db-48d1-abb4-cf47eed8631e" containerName="authorino"
Jul 28 23:42:50.402830 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402801    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1241c0e-94db-48d1-abb4-cf47eed8631e" containerName="authorino"
Jul 28 23:42:50.402923 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402908    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a1dd6e4-bc19-4e53-99c0-c29b07179cb2" containerName="manager"
Jul 28 23:42:50.402923 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.402915    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1241c0e-94db-48d1-abb4-cf47eed8631e" containerName="authorino"
Jul 28 23:42:50.407209 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.407189    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.409895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.409878    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z6vrk\""
Jul 28 23:42:50.418642 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.418620    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"]
Jul 28 23:42:50.472684 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.472651    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.472867 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.472718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc9qg\" (UniqueName: \"kubernetes.io/projected/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-kube-api-access-vc9qg\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.574092 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.574052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.574259 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.574126    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc9qg\" (UniqueName: \"kubernetes.io/projected/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-kube-api-access-vc9qg\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.574505 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.574482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.584298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.584268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc9qg\" (UniqueName: \"kubernetes.io/projected/37f2bd5c-f48b-45c4-a8c2-eb8d15f923db-kube-api-access-vc9qg\") pod \"kuadrant-operator-controller-manager-86878c769f-q4dg5\" (UID: \"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.719647 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.719553    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:50.862985 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:50.862919    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"]
Jul 28 23:42:50.867091 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:42:50.867059    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f2bd5c_f48b_45c4_a8c2_eb8d15f923db.slice/crio-b51500d7c89a5f405176154240a2345635e6af411b1bd313b5dd96d0437563ea WatchSource:0}: Error finding container b51500d7c89a5f405176154240a2345635e6af411b1bd313b5dd96d0437563ea: Status 404 returned error can't find the container with id b51500d7c89a5f405176154240a2345635e6af411b1bd313b5dd96d0437563ea
Jul 28 23:42:51.178417 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:51.178378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5" event={"ID":"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db","Type":"ContainerStarted","Data":"6c968e348948574614fd3e5017a674ea8f2210f264c830f1298946d267107cb4"}
Jul 28 23:42:51.178719 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:51.178439    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5" event={"ID":"37f2bd5c-f48b-45c4-a8c2-eb8d15f923db","Type":"ContainerStarted","Data":"b51500d7c89a5f405176154240a2345635e6af411b1bd313b5dd96d0437563ea"}
Jul 28 23:42:51.178719 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:51.178480    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:42:51.198076 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:51.198016    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5" podStartSLOduration=1.197997404 podStartE2EDuration="1.197997404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:42:51.196799837 +0000 UTC m=+1271.723849099" watchObservedRunningTime="2026-07-28 23:42:51.197997404 +0000 UTC m=+1271.725046689"
Jul 28 23:42:58.031269 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:42:58.031213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:43:02.188109 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:43:02.188073    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4dg5"
Jul 28 23:45:00.134941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.134910    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"]
Jul 28 23:45:00.138812 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.138789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"
Jul 28 23:45:00.141256 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.141228    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:45:00.149657 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.149631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"
Jul 28 23:45:00.150472 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.150450    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"]
Jul 28 23:45:00.298404 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.298377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"]
Jul 28 23:45:00.300212 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:45:00.300177    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a13a59_c6f1_4d66_bd21_8dbf9d95033e.slice/crio-34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11 WatchSource:0}: Error finding container 34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11: Status 404 returned error can't find the container with id 34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11
Jul 28 23:45:00.302283 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.302261    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:45:00.933120 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:00.933086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9" event={"ID":"19a13a59-c6f1-4d66-bd21-8dbf9d95033e","Type":"ContainerStarted","Data":"34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11"}
Jul 28 23:45:03.955008 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:03.954959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9" event={"ID":"19a13a59-c6f1-4d66-bd21-8dbf9d95033e","Type":"ContainerStarted","Data":"1d12d58fac4439a13d171fa98b9ecf63583ce34f2a77cd858323ca9b4c9642f1"}
Jul 28 23:45:03.971212 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:03.971150    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9" podStartSLOduration=1.2868768560000001 podStartE2EDuration="3.971127567s" totalImagesPullingTime="2.684250711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:45:03.968914159 +0000 UTC m=+1404.495963422" watchObservedRunningTime="2026-07-28 23:45:03.971127567 +0000 UTC m=+1404.498176831"
Jul 28 23:45:04.962883 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:04.962843    2564 generic.go:356] "Generic (PLEG): container finished" podID="19a13a59-c6f1-4d66-bd21-8dbf9d95033e" containerID="1d12d58fac4439a13d171fa98b9ecf63583ce34f2a77cd858323ca9b4c9642f1" exitCode=0
Jul 28 23:45:04.963277 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:04.962930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9" event={"ID":"19a13a59-c6f1-4d66-bd21-8dbf9d95033e","Type":"ContainerDied","Data":"1d12d58fac4439a13d171fa98b9ecf63583ce34f2a77cd858323ca9b4c9642f1"}
Jul 28 23:45:06.110848 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:06.110821    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"
Jul 28 23:45:06.976145 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:06.976117    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9"
Jul 28 23:45:06.976326 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:06.976168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754705-52lc9" event={"ID":"19a13a59-c6f1-4d66-bd21-8dbf9d95033e","Type":"ContainerDied","Data":"34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11"}
Jul 28 23:45:06.976326 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:06.976221    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34e91833265ee192dfb925772c039b3d49d3e5fad1d4effee839f672f53b3e11"
Jul 28 23:45:53.856218 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:45:53.856138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:46:20.845818 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:46:20.845777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:46:40.079817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:46:40.079788    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:46:40.082669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:46:40.082647    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:46:40.086183 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:46:40.086162    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:46:40.089080 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:46:40.089060    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:47:09.910827 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:47:09.910794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:47:10.728337 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:47:10.728300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:47:17.165186 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:47:17.165144    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:47:17.213874 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:47:17.213838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:48:08.539201 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:48:08.539160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:30.798273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:30.797835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:52.306268 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:52.306224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:53.471298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:53.471258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:53.573446 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:53.573408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:55.215442 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:55.215406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:55.286533 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:55.286480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:50:55.660823 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:50:55.660779    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:01.254239 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:01.254198    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:01.335970 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:01.335934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:01.484900 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:01.484811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:01.818762 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:01.818709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:02.084510 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:02.084384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:04.385837 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:04.385793    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:04.498628 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:04.498581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:04.593615 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:04.593577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:04.845393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:04.845320    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:05.141842 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.141751    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:05.228656 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.228621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:05.233301 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.233264    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:05.233749 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.233735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a13a59-c6f1-4d66-bd21-8dbf9d95033e" containerName="cleanup"
Jul 28 23:51:05.233801 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.233750    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a13a59-c6f1-4d66-bd21-8dbf9d95033e" containerName="cleanup"
Jul 28 23:51:05.233838 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.233823    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a13a59-c6f1-4d66-bd21-8dbf9d95033e" containerName="cleanup"
Jul 28 23:51:05.236987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.236971    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.239538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.239514    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cccd540e\""
Jul 28 23:51:05.248396 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.248368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:05.270247 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.270222    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:05.273866 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.273851    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.278465 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.278443    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cccd540e-dockercfg-srfsx\""
Jul 28 23:51:05.283698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.283675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:05.329452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.329416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.329652 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.329583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vwl\" (UniqueName: \"kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.430398 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.430279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vwl\" (UniqueName: \"kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.430398 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.430367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.430975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.430401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.430975 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.430446    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm4kw\" (UniqueName: \"kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.431332 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.431307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.438033 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.438007    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vwl\" (UniqueName: \"kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl\") pod \"payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.530969 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.530934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.531143 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.530990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4kw\" (UniqueName: \"kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.531774 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.531751    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.541264 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.541234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4kw\" (UniqueName: \"kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw\") pod \"payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") " pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.549002 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.548970    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:05.585968 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.585936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:05.712722 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.712682    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:05.714508 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:51:05.714473    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda613cb30_939c_4dbe_a625_28fe60db537a.slice/crio-941c03237b6da9dd606833219c36896ba8508c0e4fe84dfccc9fb0e62166cea2 WatchSource:0}: Error finding container 941c03237b6da9dd606833219c36896ba8508c0e4fe84dfccc9fb0e62166cea2: Status 404 returned error can't find the container with id 941c03237b6da9dd606833219c36896ba8508c0e4fe84dfccc9fb0e62166cea2
Jul 28 23:51:05.743862 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:05.743832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:05.748481 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:51:05.748454    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73bdf53_2690_4fe3_9a18_b872c27b6e4b.slice/crio-cef4052029012e2eb053654b42cafe29492e00b0c23924aaad6ae89fa8159e79 WatchSource:0}: Error finding container cef4052029012e2eb053654b42cafe29492e00b0c23924aaad6ae89fa8159e79: Status 404 returned error can't find the container with id cef4052029012e2eb053654b42cafe29492e00b0c23924aaad6ae89fa8159e79
Jul 28 23:51:06.124697 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.124660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" event={"ID":"b73bdf53-2690-4fe3-9a18-b872c27b6e4b","Type":"ContainerStarted","Data":"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"}
Jul 28 23:51:06.124905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.124714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" event={"ID":"b73bdf53-2690-4fe3-9a18-b872c27b6e4b","Type":"ContainerStarted","Data":"cef4052029012e2eb053654b42cafe29492e00b0c23924aaad6ae89fa8159e79"}
Jul 28 23:51:06.124905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.124837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:06.126403 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.126374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" event={"ID":"a613cb30-939c-4dbe-a625-28fe60db537a","Type":"ContainerStarted","Data":"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"}
Jul 28 23:51:06.126498 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.126436    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:06.126498 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.126461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" event={"ID":"a613cb30-939c-4dbe-a625-28fe60db537a","Type":"ContainerStarted","Data":"941c03237b6da9dd606833219c36896ba8508c0e4fe84dfccc9fb0e62166cea2"}
Jul 28 23:51:06.140413 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.140370    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" podStartSLOduration=1.1403381320000001 podStartE2EDuration="1.140338132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:06.139507053 +0000 UTC m=+1766.666556317" watchObservedRunningTime="2026-07-28 23:51:06.140338132 +0000 UTC m=+1766.667387394"
Jul 28 23:51:06.155432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:06.155385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" podStartSLOduration=1.155370971 podStartE2EDuration="1.155370971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:06.15323543 +0000 UTC m=+1766.680284695" watchObservedRunningTime="2026-07-28 23:51:06.155370971 +0000 UTC m=+1766.682420252"
Jul 28 23:51:17.138131 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:17.138098    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:17.138535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:17.138174    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:29.911006 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:29.910967    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"]
Jul 28 23:51:29.914810 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:29.914791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:29.917964 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:29.917476    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:51:29.920603 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:29.920575    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"]
Jul 28 23:51:29.972995 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:29.972954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx\" (UID: \"f6e4113e-3c24-45e2-aafc-689241ef0431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:30.073824 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:30.073783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx\" (UID: \"f6e4113e-3c24-45e2-aafc-689241ef0431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:30.074470 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:30.074449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx\" (UID: \"f6e4113e-3c24-45e2-aafc-689241ef0431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:30.227748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:30.227657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:30.370881 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:30.370854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"]
Jul 28 23:51:30.372140 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:51:30.372116    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e4113e_3c24_45e2_aafc_689241ef0431.slice/crio-f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a WatchSource:0}: Error finding container f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a: Status 404 returned error can't find the container with id f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a
Jul 28 23:51:31.280734 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:31.280697    2564 generic.go:356] "Generic (PLEG): container finished" podID="f6e4113e-3c24-45e2-aafc-689241ef0431" containerID="8de5013878ca013508df5511c430427844f85c6a558d2febc1ee70838d6327de" exitCode=0
Jul 28 23:51:31.281131 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:31.280786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx" event={"ID":"f6e4113e-3c24-45e2-aafc-689241ef0431","Type":"ContainerDied","Data":"8de5013878ca013508df5511c430427844f85c6a558d2febc1ee70838d6327de"}
Jul 28 23:51:31.281131 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:31.280838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx" event={"ID":"f6e4113e-3c24-45e2-aafc-689241ef0431","Type":"ContainerStarted","Data":"f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a"}
Jul 28 23:51:32.430211 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:32.430181    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:32.499067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:32.499035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca\") pod \"f6e4113e-3c24-45e2-aafc-689241ef0431\" (UID: \"f6e4113e-3c24-45e2-aafc-689241ef0431\") "
Jul 28 23:51:32.499462 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:32.499442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6e4113e-3c24-45e2-aafc-689241ef0431" (UID: "f6e4113e-3c24-45e2-aafc-689241ef0431"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:51:32.600744 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:32.600710    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e4113e-3c24-45e2-aafc-689241ef0431-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:51:33.294636 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:33.294606    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"
Jul 28 23:51:33.294884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:33.294636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx" event={"ID":"f6e4113e-3c24-45e2-aafc-689241ef0431","Type":"ContainerDied","Data":"f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a"}
Jul 28 23:51:33.294884 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:33.294692    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9330dafcdc2444e44f6119449912b9f9216d72b1de7a969b294cd5f3c7ae88a"
Jul 28 23:51:35.196757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.196720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.221140 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.221093    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:35.221669 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.221633    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" podUID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" containerName="payload-processing" containerID="cri-o://6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279" gracePeriod=30
Jul 28 23:51:35.230739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.230405    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:35.230739 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.230644    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" podUID="a613cb30-939c-4dbe-a625-28fe60db537a" containerName="payload-pre-processing" containerID="cri-o://3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368" gracePeriod=30
Jul 28 23:51:35.261859 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.261779    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.322143 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.322107    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.353699 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.353660    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"]
Jul 28 23:51:35.356232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.356206    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cccd540e-4j8kx"]
Jul 28 23:51:35.378096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.377961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.499607 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.499540    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.500968 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.500949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:35.549845 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.549822    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:35.632154 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632122    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume\") pod \"a613cb30-939c-4dbe-a625-28fe60db537a\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") "
Jul 28 23:51:35.632334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume\") pod \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") "
Jul 28 23:51:35.632334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632196    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69vwl\" (UniqueName: \"kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl\") pod \"a613cb30-939c-4dbe-a625-28fe60db537a\" (UID: \"a613cb30-939c-4dbe-a625-28fe60db537a\") "
Jul 28 23:51:35.632334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632287    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm4kw\" (UniqueName: \"kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw\") pod \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\" (UID: \"b73bdf53-2690-4fe3-9a18-b872c27b6e4b\") "
Jul 28 23:51:35.632719 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632680    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a613cb30-939c-4dbe-a625-28fe60db537a" (UID: "a613cb30-939c-4dbe-a625-28fe60db537a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:51:35.632850 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.632732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b73bdf53-2690-4fe3-9a18-b872c27b6e4b" (UID: "b73bdf53-2690-4fe3-9a18-b872c27b6e4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:51:35.634376 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.634354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw" (OuterVolumeSpecName: "kube-api-access-qm4kw") pod "b73bdf53-2690-4fe3-9a18-b872c27b6e4b" (UID: "b73bdf53-2690-4fe3-9a18-b872c27b6e4b"). InnerVolumeSpecName "kube-api-access-qm4kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:51:35.634462 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.634390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl" (OuterVolumeSpecName: "kube-api-access-69vwl") pod "a613cb30-939c-4dbe-a625-28fe60db537a" (UID: "a613cb30-939c-4dbe-a625-28fe60db537a"). InnerVolumeSpecName "kube-api-access-69vwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:51:35.720024 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.719918    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:35.733313 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.733285    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a613cb30-939c-4dbe-a625-28fe60db537a-plugins-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.733313 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.733310    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-plugins-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.733474 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.733320    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69vwl\" (UniqueName: \"kubernetes.io/projected/a613cb30-939c-4dbe-a625-28fe60db537a-kube-api-access-69vwl\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.733474 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.733328    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm4kw\" (UniqueName: \"kubernetes.io/projected/b73bdf53-2690-4fe3-9a18-b872c27b6e4b-kube-api-access-qm4kw\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.969812 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:35.969751    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:36.071329 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.071287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:36.116612 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.116577    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e4113e-3c24-45e2-aafc-689241ef0431" path="/var/lib/kubelet/pods/f6e4113e-3c24-45e2-aafc-689241ef0431/volumes"
Jul 28 23:51:36.180991 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.180944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:36.313590 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.313554    2564 generic.go:356] "Generic (PLEG): container finished" podID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" containerID="6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279" exitCode=0
Jul 28 23:51:36.314010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.313611    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"
Jul 28 23:51:36.314010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.313643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" event={"ID":"b73bdf53-2690-4fe3-9a18-b872c27b6e4b","Type":"ContainerDied","Data":"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"}
Jul 28 23:51:36.314010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.313692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6" event={"ID":"b73bdf53-2690-4fe3-9a18-b872c27b6e4b","Type":"ContainerDied","Data":"cef4052029012e2eb053654b42cafe29492e00b0c23924aaad6ae89fa8159e79"}
Jul 28 23:51:36.314010 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.313721    2564 scope.go:117] "RemoveContainer" containerID="6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"
Jul 28 23:51:36.315199 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.315175    2564 generic.go:356] "Generic (PLEG): container finished" podID="a613cb30-939c-4dbe-a625-28fe60db537a" containerID="3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368" exitCode=0
Jul 28 23:51:36.315278 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.315223    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"
Jul 28 23:51:36.315338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.315225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" event={"ID":"a613cb30-939c-4dbe-a625-28fe60db537a","Type":"ContainerDied","Data":"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"}
Jul 28 23:51:36.315444 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.315387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh" event={"ID":"a613cb30-939c-4dbe-a625-28fe60db537a","Type":"ContainerDied","Data":"941c03237b6da9dd606833219c36896ba8508c0e4fe84dfccc9fb0e62166cea2"}
Jul 28 23:51:36.325773 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.325753    2564 scope.go:117] "RemoveContainer" containerID="6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"
Jul 28 23:51:36.326053 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:51:36.326031    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279\": container with ID starting with 6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279 not found: ID does not exist" containerID="6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"
Jul 28 23:51:36.326117 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.326060    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279"} err="failed to get container status \"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279\": rpc error: code = NotFound desc = could not find container \"6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279\": container with ID starting with 6f2efe1950e5a7ce58c425bceb4192eff4571774fec06b8c89265a16d9a30279 not found: ID does not exist"
Jul 28 23:51:36.326117 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.326078    2564 scope.go:117] "RemoveContainer" containerID="3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"
Jul 28 23:51:36.334690 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.334566    2564 scope.go:117] "RemoveContainer" containerID="3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"
Jul 28 23:51:36.334886 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:51:36.334857    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368\": container with ID starting with 3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368 not found: ID does not exist" containerID="3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"
Jul 28 23:51:36.335019 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.334893    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368"} err="failed to get container status \"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368\": rpc error: code = NotFound desc = could not find container \"3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368\": container with ID starting with 3e7b3325831bc54430f691aff78496673ea15c3128819e60ae25ec447b2f0368 not found: ID does not exist"
Jul 28 23:51:36.335302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.335285    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:36.337661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.337642    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cccd540e-6bbd9fd7cd-j5mr6"]
Jul 28 23:51:36.346834 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.346815    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:36.360031 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.360009    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cccd540e-56fc558f56-bvnjh"]
Jul 28 23:51:36.477725 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:36.477686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:37.022148 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:37.022109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:38.123216 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.123181    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a613cb30-939c-4dbe-a625-28fe60db537a" path="/var/lib/kubelet/pods/a613cb30-939c-4dbe-a625-28fe60db537a/volumes"
Jul 28 23:51:38.123617 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.123583    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" path="/var/lib/kubelet/pods/b73bdf53-2690-4fe3-9a18-b872c27b6e4b/volumes"
Jul 28 23:51:38.701268 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.701234    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:38.778447 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778415    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:51:38.778856 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778844    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a613cb30-939c-4dbe-a625-28fe60db537a" containerName="payload-pre-processing"
Jul 28 23:51:38.778902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778858    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a613cb30-939c-4dbe-a625-28fe60db537a" containerName="payload-pre-processing"
Jul 28 23:51:38.778902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778865    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e4113e-3c24-45e2-aafc-689241ef0431" containerName="revoke-keys"
Jul 28 23:51:38.778902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778871    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e4113e-3c24-45e2-aafc-689241ef0431" containerName="revoke-keys"
Jul 28 23:51:38.778902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778886    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" containerName="payload-processing"
Jul 28 23:51:38.778902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778892    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" containerName="payload-processing"
Jul 28 23:51:38.779093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778953    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a613cb30-939c-4dbe-a625-28fe60db537a" containerName="payload-pre-processing"
Jul 28 23:51:38.779093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778961    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e4113e-3c24-45e2-aafc-689241ef0431" containerName="revoke-keys"
Jul 28 23:51:38.779093 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.778970    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73bdf53-2690-4fe3-9a18-b872c27b6e4b" containerName="payload-processing"
Jul 28 23:51:38.782221 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.782199    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.785233 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.785208    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-47ffb609-serving-cert\""
Jul 28 23:51:38.785386 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.785235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 28 23:51:38.792678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.792491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:51:38.813811 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.813772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:38.866934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.866895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv54h\" (UniqueName: \"kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.867120 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.866951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.867120 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.867040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.964168 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.964062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:38.967850 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.967817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.967994 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.967913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv54h\" (UniqueName: \"kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.967994 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.967935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.968538 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.968515    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.970320 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.970301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:38.976506 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:38.976485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv54h\" (UniqueName: \"kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h\") pod \"maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:39.097663 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.097627    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:39.245426 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.245397    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:51:39.246528 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:51:39.246495    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de5498b_6467_414b_a64f_3e5a07dc4aaa.slice/crio-ae809b6e5dc3d9170cbcaa56b2a5502ac827eab94255e3a0569989310fe9b360 WatchSource:0}: Error finding container ae809b6e5dc3d9170cbcaa56b2a5502ac827eab94255e3a0569989310fe9b360: Status 404 returned error can't find the container with id ae809b6e5dc3d9170cbcaa56b2a5502ac827eab94255e3a0569989310fe9b360
Jul 28 23:51:39.247897 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.247876    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:51:39.327164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.327122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:39.335969 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.335940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" event={"ID":"9de5498b-6467-414b-a64f-3e5a07dc4aaa","Type":"ContainerStarted","Data":"ae809b6e5dc3d9170cbcaa56b2a5502ac827eab94255e3a0569989310fe9b360"}
Jul 28 23:51:39.425014 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.424981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:51:39.469519 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.469485    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:51:39.474563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.474539    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.477071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.477049    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-47ffb609\""
Jul 28 23:51:39.477191 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.477077    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-47ffb609-dockercfg-hf5nf\""
Jul 28 23:51:39.483399 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.483335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:51:39.502240 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.502153    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:51:39.505724 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.505704    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.513451 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.513423    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:51:39.573433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.573395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.573433 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.573439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4896z\" (UniqueName: \"kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.573631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.573485    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjw2g\" (UniqueName: \"kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.573631 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.573503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.674096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjw2g\" (UniqueName: \"kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.674096 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.674298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.674298 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4896z\" (UniqueName: \"kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.674956 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674935    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.675001 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.674957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.681996 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.681967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4896z\" (UniqueName: \"kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z\") pod \"payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:39.682142 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.682125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjw2g\" (UniqueName: \"kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g\") pod \"payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") " pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.787278 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.787195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:39.818025 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:39.817984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:40.008307 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.007913    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:51:40.010803 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:51:40.010772    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49696715_035e_49a1_af3e_c0b03b2d0adf.slice/crio-ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf WatchSource:0}: Error finding container ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf: Status 404 returned error can't find the container with id ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf
Jul 28 23:51:40.044817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.044785    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:51:40.150277 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.150106    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:51:40.154811 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.154784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:51:40.166361 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.163920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:51:40.170891 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.170867    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:51:40.353852 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.353757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" event={"ID":"8ca1b8e0-55d6-465d-a322-f55ce7934eff","Type":"ContainerStarted","Data":"9f101ab813a54c666972312429c22be0b6b7aed967069bb132b93f1692f92c35"}
Jul 28 23:51:40.354367 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.353875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" event={"ID":"8ca1b8e0-55d6-465d-a322-f55ce7934eff","Type":"ContainerStarted","Data":"8dc55e011587072e3ad3d7e7fffdd1917ba2ec8c73a7f89febf7d8b4ad8b4d15"}
Jul 28 23:51:40.354367 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.354224    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:40.356296 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.356267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" event={"ID":"49696715-035e-49a1-af3e-c0b03b2d0adf","Type":"ContainerStarted","Data":"7e8caa98c29670c98d9aa5b59fc847386a64e6d3f9e6abdda0f27d8658a3eaae"}
Jul 28 23:51:40.356449 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.356327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" event={"ID":"49696715-035e-49a1-af3e-c0b03b2d0adf","Type":"ContainerStarted","Data":"ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf"}
Jul 28 23:51:40.356449 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.356371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:51:40.371808 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.371734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" podStartSLOduration=1.3717133989999999 podStartE2EDuration="1.371713399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:40.368385735 +0000 UTC m=+1800.895434998" watchObservedRunningTime="2026-07-28 23:51:40.371713399 +0000 UTC m=+1800.898762661"
Jul 28 23:51:40.387319 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:40.387272    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" podStartSLOduration=1.387256757 podStartE2EDuration="1.387256757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:40.384696334 +0000 UTC m=+1800.911745599" watchObservedRunningTime="2026-07-28 23:51:40.387256757 +0000 UTC m=+1800.914306020"
Jul 28 23:51:42.370387 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:42.370332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" event={"ID":"9de5498b-6467-414b-a64f-3e5a07dc4aaa","Type":"ContainerStarted","Data":"34fcba16a38b25543f52b8f3f6e152232b44bc8c78391eefba4d12ec1872ba14"}
Jul 28 23:51:42.370825 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:42.370417    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:42.390548 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:42.390491    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" podStartSLOduration=1.856359586 podStartE2EDuration="4.390471501s" totalImagesPullingTime="2.534111915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:42.388031185 +0000 UTC m=+1802.915080448" watchObservedRunningTime="2026-07-28 23:51:42.390471501 +0000 UTC m=+1802.917520765"
Jul 28 23:51:48.383496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:48.383465    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:51:51.365284 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:51.365252    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:51:51.365787 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:51:51.365319    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:52:03.700022 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.699979    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"]
Jul 28 23:52:03.704037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.704012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:03.706906 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.706878    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:52:03.709997 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.709970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"]
Jul 28 23:52:03.802001 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.801957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf\" (UID: \"a60ed79c-6125-48af-baec-b4219c544667\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:03.903153 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.903111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf\" (UID: \"a60ed79c-6125-48af-baec-b4219c544667\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:03.903856 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:03.903828    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf\" (UID: \"a60ed79c-6125-48af-baec-b4219c544667\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:04.017624 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:04.017521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:04.365452 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:04.365418    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"]
Jul 28 23:52:04.366929 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:52:04.366900    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60ed79c_6125_48af_baec_b4219c544667.slice/crio-c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe WatchSource:0}: Error finding container c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe: Status 404 returned error can't find the container with id c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe
Jul 28 23:52:04.516825 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:04.516790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf" event={"ID":"a60ed79c-6125-48af-baec-b4219c544667","Type":"ContainerStarted","Data":"c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe"}
Jul 28 23:52:05.524008 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:05.523919    2564 generic.go:356] "Generic (PLEG): container finished" podID="a60ed79c-6125-48af-baec-b4219c544667" containerID="053fc3ec467bcffbf70bfd2f74aad70730d6c64dc4365018ffffdc3d99fecd7c" exitCode=0
Jul 28 23:52:05.524387 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:05.524005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf" event={"ID":"a60ed79c-6125-48af-baec-b4219c544667","Type":"ContainerDied","Data":"053fc3ec467bcffbf70bfd2f74aad70730d6c64dc4365018ffffdc3d99fecd7c"}
Jul 28 23:52:06.677753 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:06.677728    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:06.832426 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:06.832396    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca\") pod \"a60ed79c-6125-48af-baec-b4219c544667\" (UID: \"a60ed79c-6125-48af-baec-b4219c544667\") "
Jul 28 23:52:06.832769 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:06.832746    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a60ed79c-6125-48af-baec-b4219c544667" (UID: "a60ed79c-6125-48af-baec-b4219c544667"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:06.934115 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:06.934077    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60ed79c-6125-48af-baec-b4219c544667-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:07.538518 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:07.538480    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"
Jul 28 23:52:07.538518 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:07.538500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf" event={"ID":"a60ed79c-6125-48af-baec-b4219c544667","Type":"ContainerDied","Data":"c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe"}
Jul 28 23:52:07.538736 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:07.538546    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c124add7a296855462784845b6e8a240198d66a5c1f03a1674efb0a98c357efe"
Jul 28 23:52:10.212270 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.212225    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:52:10.212937 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.212855    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" podUID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" containerName="maas-api" containerID="cri-o://34fcba16a38b25543f52b8f3f6e152232b44bc8c78391eefba4d12ec1872ba14" gracePeriod=30
Jul 28 23:52:10.246987 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.246954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.275557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.275524    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:52:10.275861 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.275828    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" podUID="49696715-035e-49a1-af3e-c0b03b2d0adf" containerName="payload-processing" containerID="cri-o://7e8caa98c29670c98d9aa5b59fc847386a64e6d3f9e6abdda0f27d8658a3eaae" gracePeriod=30
Jul 28 23:52:10.286759 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.286729    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:52:10.287123 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.286941    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" podUID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" containerName="payload-pre-processing" containerID="cri-o://9f101ab813a54c666972312429c22be0b6b7aed967069bb132b93f1692f92c35" gracePeriod=30
Jul 28 23:52:10.326824 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.326794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.396421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.396283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.433331 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.433289    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"]
Jul 28 23:52:10.439273 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.439236    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-47ffb609-crmsf"]
Jul 28 23:52:10.475075 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.474933    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.570384 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.568509    2564 generic.go:356] "Generic (PLEG): container finished" podID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" containerID="34fcba16a38b25543f52b8f3f6e152232b44bc8c78391eefba4d12ec1872ba14" exitCode=0
Jul 28 23:52:10.570384 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.568620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" event={"ID":"9de5498b-6467-414b-a64f-3e5a07dc4aaa","Type":"ContainerDied","Data":"34fcba16a38b25543f52b8f3f6e152232b44bc8c78391eefba4d12ec1872ba14"}
Jul 28 23:52:10.572726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.572192    2564 generic.go:356] "Generic (PLEG): container finished" podID="49696715-035e-49a1-af3e-c0b03b2d0adf" containerID="7e8caa98c29670c98d9aa5b59fc847386a64e6d3f9e6abdda0f27d8658a3eaae" exitCode=0
Jul 28 23:52:10.572726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.572270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" event={"ID":"49696715-035e-49a1-af3e-c0b03b2d0adf","Type":"ContainerDied","Data":"7e8caa98c29670c98d9aa5b59fc847386a64e6d3f9e6abdda0f27d8658a3eaae"}
Jul 28 23:52:10.572726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.572337    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7" event={"ID":"49696715-035e-49a1-af3e-c0b03b2d0adf","Type":"ContainerDied","Data":"ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf"}
Jul 28 23:52:10.572726 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.572381    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea18dc4bbae9b07c2c9607958590aa7d54d4ee63bc01287a2cb873ffa70d9dcf"
Jul 28 23:52:10.576536 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.576274    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" containerID="9f101ab813a54c666972312429c22be0b6b7aed967069bb132b93f1692f92c35" exitCode=0
Jul 28 23:52:10.576536 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.576376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" event={"ID":"8ca1b8e0-55d6-465d-a322-f55ce7934eff","Type":"ContainerDied","Data":"9f101ab813a54c666972312429c22be0b6b7aed967069bb132b93f1692f92c35"}
Jul 28 23:52:10.583498 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.583466    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:52:10.588994 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.588955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.607585 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.607550    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:52:10.627338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.627311    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:52:10.773263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773181    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjw2g\" (UniqueName: \"kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g\") pod \"49696715-035e-49a1-af3e-c0b03b2d0adf\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") "
Jul 28 23:52:10.773263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773223    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume\") pod \"49696715-035e-49a1-af3e-c0b03b2d0adf\" (UID: \"49696715-035e-49a1-af3e-c0b03b2d0adf\") "
Jul 28 23:52:10.773263 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls\") pod \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") "
Jul 28 23:52:10.773592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca\") pod \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") "
Jul 28 23:52:10.773592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773406    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume\") pod \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") "
Jul 28 23:52:10.773592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773437    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv54h\" (UniqueName: \"kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h\") pod \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\" (UID: \"9de5498b-6467-414b-a64f-3e5a07dc4aaa\") "
Jul 28 23:52:10.773592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773464    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4896z\" (UniqueName: \"kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z\") pod \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\" (UID: \"8ca1b8e0-55d6-465d-a322-f55ce7934eff\") "
Jul 28 23:52:10.773806 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773746    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49696715-035e-49a1-af3e-c0b03b2d0adf" (UID: "49696715-035e-49a1-af3e-c0b03b2d0adf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:10.774002 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.773936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9de5498b-6467-414b-a64f-3e5a07dc4aaa" (UID: "9de5498b-6467-414b-a64f-3e5a07dc4aaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:10.774139 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.774019    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ca1b8e0-55d6-465d-a322-f55ce7934eff" (UID: "8ca1b8e0-55d6-465d-a322-f55ce7934eff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:10.775688 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.775660    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g" (OuterVolumeSpecName: "kube-api-access-bjw2g") pod "49696715-035e-49a1-af3e-c0b03b2d0adf" (UID: "49696715-035e-49a1-af3e-c0b03b2d0adf"). InnerVolumeSpecName "kube-api-access-bjw2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:10.775791 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.775695    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z" (OuterVolumeSpecName: "kube-api-access-4896z") pod "8ca1b8e0-55d6-465d-a322-f55ce7934eff" (UID: "8ca1b8e0-55d6-465d-a322-f55ce7934eff"). InnerVolumeSpecName "kube-api-access-4896z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:10.775791 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.775717    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h" (OuterVolumeSpecName: "kube-api-access-vv54h") pod "9de5498b-6467-414b-a64f-3e5a07dc4aaa" (UID: "9de5498b-6467-414b-a64f-3e5a07dc4aaa"). InnerVolumeSpecName "kube-api-access-vv54h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:10.775791 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.775772    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9de5498b-6467-414b-a64f-3e5a07dc4aaa" (UID: "9de5498b-6467-414b-a64f-3e5a07dc4aaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:52:10.821833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.821798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:10.874232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874192    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de5498b-6467-414b-a64f-3e5a07dc4aaa-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874224    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca1b8e0-55d6-465d-a322-f55ce7934eff-plugins-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874232 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874240    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv54h\" (UniqueName: \"kubernetes.io/projected/9de5498b-6467-414b-a64f-3e5a07dc4aaa-kube-api-access-vv54h\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874486 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874252    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4896z\" (UniqueName: \"kubernetes.io/projected/8ca1b8e0-55d6-465d-a322-f55ce7934eff-kube-api-access-4896z\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874486 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874266    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjw2g\" (UniqueName: \"kubernetes.io/projected/49696715-035e-49a1-af3e-c0b03b2d0adf-kube-api-access-bjw2g\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874486 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874277    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49696715-035e-49a1-af3e-c0b03b2d0adf-plugins-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:10.874486 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:10.874290    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9de5498b-6467-414b-a64f-3e5a07dc4aaa-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:11.045062 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.045031    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:11.328576 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328494    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328935    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a60ed79c-6125-48af-baec-b4219c544667" containerName="revoke-keys"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328947    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a60ed79c-6125-48af-baec-b4219c544667" containerName="revoke-keys"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328958    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49696715-035e-49a1-af3e-c0b03b2d0adf" containerName="payload-processing"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328965    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="49696715-035e-49a1-af3e-c0b03b2d0adf" containerName="payload-processing"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328991    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" containerName="payload-pre-processing"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.328997    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" containerName="payload-pre-processing"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329015    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" containerName="maas-api"
Jul 28 23:52:11.329015 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329021    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" containerName="maas-api"
Jul 28 23:52:11.329450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329096    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" containerName="payload-pre-processing"
Jul 28 23:52:11.329450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329104    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a60ed79c-6125-48af-baec-b4219c544667" containerName="revoke-keys"
Jul 28 23:52:11.329450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329111    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" containerName="maas-api"
Jul 28 23:52:11.329450 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.329118    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="49696715-035e-49a1-af3e-c0b03b2d0adf" containerName="payload-processing"
Jul 28 23:52:11.332573 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.332553    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.337709 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.337690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2aaf48a6-openshift-default-dockercfg-cvj7x\""
Jul 28 23:52:11.348983 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.348960    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:52:11.365896 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.365860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:11.378253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378428 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378428 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378428 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378517    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vncr\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.378700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.378564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.479700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vncr\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.479868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.479868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.479868 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.479988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480053 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480285 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480503 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480582 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.480895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.480868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.482454 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.482430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.482625 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.482608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.487515 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.487487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vncr\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.487641 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.487622    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token\") pod \"e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") " pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.584774 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.584694    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"
Jul 28 23:52:11.584774 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.584738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c" event={"ID":"8ca1b8e0-55d6-465d-a322-f55ce7934eff","Type":"ContainerDied","Data":"8dc55e011587072e3ad3d7e7fffdd1917ba2ec8c73a7f89febf7d8b4ad8b4d15"}
Jul 28 23:52:11.585007 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.584811    2564 scope.go:117] "RemoveContainer" containerID="9f101ab813a54c666972312429c22be0b6b7aed967069bb132b93f1692f92c35"
Jul 28 23:52:11.586177 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.586151    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"
Jul 28 23:52:11.586271 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.586202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw" event={"ID":"9de5498b-6467-414b-a64f-3e5a07dc4aaa","Type":"ContainerDied","Data":"ae809b6e5dc3d9170cbcaa56b2a5502ac827eab94255e3a0569989310fe9b360"}
Jul 28 23:52:11.586413 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.586394    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"
Jul 28 23:52:11.594248 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.594230    2564 scope.go:117] "RemoveContainer" containerID="34fcba16a38b25543f52b8f3f6e152232b44bc8c78391eefba4d12ec1872ba14"
Jul 28 23:52:11.613352 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.613311    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:52:11.617094 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.617071    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47ffb609-c8b8b9cfb-7lhnw"]
Jul 28 23:52:11.627941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.627917    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:52:11.630477 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.630457    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47ffb609-57977bf9f7-fqzc7"]
Jul 28 23:52:11.640393 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.640364    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:52:11.642663 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.642640    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:11.642842 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.642824    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47ffb609-59584bf5d8-mtw6c"]
Jul 28 23:52:11.750040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.749993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:11.792389 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.792278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:52:11.794849 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:52:11.794821    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c90f3b_a2be_4856_96f1_0c25b60d3dcf.slice/crio-eb260007201aaee16077aeecd6cae3104cbdb085af6c5bee061437fe7f83be74 WatchSource:0}: Error finding container eb260007201aaee16077aeecd6cae3104cbdb085af6c5bee061437fe7f83be74: Status 404 returned error can't find the container with id eb260007201aaee16077aeecd6cae3104cbdb085af6c5bee061437fe7f83be74
Jul 28 23:52:11.797068 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.797041    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:11.797129 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.797101    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:11.797164 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:11.797132    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:12.117840 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.117762    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49696715-035e-49a1-af3e-c0b03b2d0adf" path="/var/lib/kubelet/pods/49696715-035e-49a1-af3e-c0b03b2d0adf/volumes"
Jul 28 23:52:12.118175 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.118161    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ca1b8e0-55d6-465d-a322-f55ce7934eff" path="/var/lib/kubelet/pods/8ca1b8e0-55d6-465d-a322-f55ce7934eff/volumes"
Jul 28 23:52:12.118535 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.118522    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de5498b-6467-414b-a64f-3e5a07dc4aaa" path="/var/lib/kubelet/pods/9de5498b-6467-414b-a64f-3e5a07dc4aaa/volumes"
Jul 28 23:52:12.118818 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.118807    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a60ed79c-6125-48af-baec-b4219c544667" path="/var/lib/kubelet/pods/a60ed79c-6125-48af-baec-b4219c544667/volumes"
Jul 28 23:52:12.600396 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.600354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerStarted","Data":"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"}
Jul 28 23:52:12.600855 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.600454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerStarted","Data":"eb260007201aaee16077aeecd6cae3104cbdb085af6c5bee061437fe7f83be74"}
Jul 28 23:52:12.620173 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.620119    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podStartSLOduration=1.620105211 podStartE2EDuration="1.620105211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:52:12.617690668 +0000 UTC m=+1833.144739932" watchObservedRunningTime="2026-07-28 23:52:12.620105211 +0000 UTC m=+1833.147154538"
Jul 28 23:52:12.643419 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.643390    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:12.648548 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:12.648524    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:13.609168 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:13.609135    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:13.610191 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:13.610173    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:13.977556 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:13.977475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.079129 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.079090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.184203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.184168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.334833 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.334799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.663953 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.663867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.755337 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.755302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:14.760427 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.760394    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:14.767492 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.767468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.771134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.771114    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2aaf48a6\""
Jul 28 23:52:14.771134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.771130    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2aaf48a6-dockercfg-qw68q\""
Jul 28 23:52:14.776167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.775865    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:14.811496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.811462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.811645 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.811586    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w5dg\" (UniqueName: \"kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.912813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.912769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w5dg\" (UniqueName: \"kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.913004 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.912881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.913743 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.913718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:14.940659 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:14.940591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w5dg\" (UniqueName: \"kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg\") pod \"payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:15.080704 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.080674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:15.430806 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.430774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:15.432241 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:52:15.432211    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550fcdfb_21c9_4e22_bcdf_ceec3f749273.slice/crio-30cea86cf227cc89cdbcdfc46ead0e4994eb7350aea857b7292856b3665b855d WatchSource:0}: Error finding container 30cea86cf227cc89cdbcdfc46ead0e4994eb7350aea857b7292856b3665b855d: Status 404 returned error can't find the container with id 30cea86cf227cc89cdbcdfc46ead0e4994eb7350aea857b7292856b3665b855d
Jul 28 23:52:15.622498 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.622439    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" event={"ID":"550fcdfb-21c9-4e22-bcdf-ceec3f749273","Type":"ContainerStarted","Data":"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"}
Jul 28 23:52:15.622686 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.622524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" event={"ID":"550fcdfb-21c9-4e22-bcdf-ceec3f749273","Type":"ContainerStarted","Data":"30cea86cf227cc89cdbcdfc46ead0e4994eb7350aea857b7292856b3665b855d"}
Jul 28 23:52:15.622827 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.622810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:15.641557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:15.641508    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" podStartSLOduration=1.641491463 podStartE2EDuration="1.641491463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:52:15.639434328 +0000 UTC m=+1836.166483591" watchObservedRunningTime="2026-07-28 23:52:15.641491463 +0000 UTC m=+1836.168540726"
Jul 28 23:52:26.630991 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:26.630956    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:28.611258 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.611176    2564 patch_prober.go:28] interesting pod/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 28 23:52:28.611258 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.611249    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 28 23:52:28.716460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.716373    2564 generic.go:356] "Generic (PLEG): container finished" podID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerID="e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc" exitCode=0
Jul 28 23:52:28.716460 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.716412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerDied","Data":"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"}
Jul 28 23:52:28.716817 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.716802    2564 scope.go:117] "RemoveContainer" containerID="e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"
Jul 28 23:52:28.718048 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.718016    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:28.718134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.718113    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:28.718181 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:28.718157    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:29.732168 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:29.732123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerStarted","Data":"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"}
Jul 28 23:52:30.643374 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:30.643299    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:30.643702 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:30.643678    2564 patch_prober.go:28] interesting pod/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 28 23:52:30.643757 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:30.643739    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 28 23:52:31.643047 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:31.643005    2564 patch_prober.go:28] interesting pod/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 28 23:52:31.643554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:31.643077    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 28 23:52:32.643924 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:32.643882    2564 patch_prober.go:28] interesting pod/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 28 23:52:32.644439 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:32.643942    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 28 23:52:33.646890 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:33.646859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:33.647289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:33.647265    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:33.648125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:33.648106    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:52:54.820135 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:54.820097    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:54.869827 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:54.869783    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:54.870965 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:54.870879    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" podUID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" containerName="payload-pre-processing" containerID="cri-o://4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a" gracePeriod=30
Jul 28 23:52:54.925945 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:54.925840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:55.000235 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.000199    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:55.066253 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.066156    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:55.141568 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.141533    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:55.224101 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.224069    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w5dg\" (UniqueName: \"kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg\") pod \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") "
Jul 28 23:52:55.224310 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.224116    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume\") pod \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\" (UID: \"550fcdfb-21c9-4e22-bcdf-ceec3f749273\") "
Jul 28 23:52:55.224589 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.224561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "550fcdfb-21c9-4e22-bcdf-ceec3f749273" (UID: "550fcdfb-21c9-4e22-bcdf-ceec3f749273"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:55.226200 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.226171    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg" (OuterVolumeSpecName: "kube-api-access-2w5dg") pod "550fcdfb-21c9-4e22-bcdf-ceec3f749273" (UID: "550fcdfb-21c9-4e22-bcdf-ceec3f749273"). InnerVolumeSpecName "kube-api-access-2w5dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:55.325432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.325396    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w5dg\" (UniqueName: \"kubernetes.io/projected/550fcdfb-21c9-4e22-bcdf-ceec3f749273-kube-api-access-2w5dg\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:55.325432 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.325424    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/550fcdfb-21c9-4e22-bcdf-ceec3f749273-plugins-config-volume\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:52:55.890862 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.890826    2564 generic.go:356] "Generic (PLEG): container finished" podID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" containerID="4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a" exitCode=0
Jul 28 23:52:55.891338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.890880    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"
Jul 28 23:52:55.891338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.890888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" event={"ID":"550fcdfb-21c9-4e22-bcdf-ceec3f749273","Type":"ContainerDied","Data":"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"}
Jul 28 23:52:55.891338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.890941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j" event={"ID":"550fcdfb-21c9-4e22-bcdf-ceec3f749273","Type":"ContainerDied","Data":"30cea86cf227cc89cdbcdfc46ead0e4994eb7350aea857b7292856b3665b855d"}
Jul 28 23:52:55.891338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.890971    2564 scope.go:117] "RemoveContainer" containerID="4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"
Jul 28 23:52:55.901279 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.901258    2564 scope.go:117] "RemoveContainer" containerID="4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"
Jul 28 23:52:55.901582 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:52:55.901564    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a\": container with ID starting with 4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a not found: ID does not exist" containerID="4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"
Jul 28 23:52:55.901651 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.901589    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a"} err="failed to get container status \"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a\": rpc error: code = NotFound desc = could not find container \"4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a\": container with ID starting with 4792fe14b4f512756aec732d9ee164fae716d7616c7e1838b256e0b1378f9a7a not found: ID does not exist"
Jul 28 23:52:55.916071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.916037    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:55.920130 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:55.920100    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2aaf48a6-58c694cdb4-cjx6j"]
Jul 28 23:52:56.121550 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.121512    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" path="/var/lib/kubelet/pods/550fcdfb-21c9-4e22-bcdf-ceec3f749273/volumes"
Jul 28 23:52:56.146623 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.146530    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:52:56.146803 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.146748    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy" containerID="cri-o://ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be" gracePeriod=30
Jul 28 23:52:56.167560 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.167511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:56.378329 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.378294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:56.578647 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.578610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:56.869985 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:56.869887    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:57.121286 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:57.121195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:58.931856 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:58.931817    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:59.014327 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:59.014290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:59.084604 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:59.084556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:59.301529 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:59.301497    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:59.608627 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:59.608542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:52:59.707159 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:52:59.707130    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:01.415496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.415432    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:53:01.484820 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.484788    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.484867    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.484902    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.484936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vncr\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.484961    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485008    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485064    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485098    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485161    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert\") pod \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\" (UID: \"48c90f3b-a2be-4856-96f1-0c25b60d3dcf\") "
Jul 28 23:53:01.485289 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485221    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:01.485540 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485392    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:01.485625 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485604    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.485683 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485634    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.485683 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485657    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:53:01.485945 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.485919    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:01.486302 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.486275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:01.487246 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.487218    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:53:01.487421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.487279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:53:01.487421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.487318    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:01.487421 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.487283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr" (OuterVolumeSpecName: "kube-api-access-4vncr") pod "48c90f3b-a2be-4856-96f1-0c25b60d3dcf" (UID: "48c90f3b-a2be-4856-96f1-0c25b60d3dcf"). InnerVolumeSpecName "kube-api-access-4vncr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:53:01.586407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586333    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586402    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586411    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586407 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586419    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586428    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586440    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vncr\" (UniqueName: \"kubernetes.io/projected/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-kube-api-access-4vncr\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.586678 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.586450    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48c90f3b-a2be-4856-96f1-0c25b60d3dcf-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:01.930567 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.930526    2564 generic.go:356] "Generic (PLEG): container finished" podID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerID="ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be" exitCode=0
Jul 28 23:53:01.930747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.930585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"
Jul 28 23:53:01.930747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.930595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerDied","Data":"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"}
Jul 28 23:53:01.930747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.930660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4" event={"ID":"48c90f3b-a2be-4856-96f1-0c25b60d3dcf","Type":"ContainerDied","Data":"eb260007201aaee16077aeecd6cae3104cbdb085af6c5bee061437fe7f83be74"}
Jul 28 23:53:01.930747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.930686    2564 scope.go:117] "RemoveContainer" containerID="ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"
Jul 28 23:53:01.941554 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.941535    2564 scope.go:117] "RemoveContainer" containerID="e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"
Jul 28 23:53:01.950476 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.950452    2564 scope.go:117] "RemoveContainer" containerID="ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"
Jul 28 23:53:01.950775 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:53:01.950753    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be\": container with ID starting with ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be not found: ID does not exist" containerID="ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"
Jul 28 23:53:01.950867 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.950788    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be"} err="failed to get container status \"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be\": rpc error: code = NotFound desc = could not find container \"ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be\": container with ID starting with ce5b522b73d7de29f0e7d8d5211642f81acbfdc9c63a1f03259b7c38eda222be not found: ID does not exist"
Jul 28 23:53:01.950867 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.950811    2564 scope.go:117] "RemoveContainer" containerID="e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"
Jul 28 23:53:01.951112 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:53:01.951091    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc\": container with ID starting with e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc not found: ID does not exist" containerID="e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"
Jul 28 23:53:01.951152 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.951119    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc"} err="failed to get container status \"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc\": rpc error: code = NotFound desc = could not find container \"e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc\": container with ID starting with e2bacfb329baeb370f680c59c0d39d913089699051435bd219407e7b7a2d71cc not found: ID does not exist"
Jul 28 23:53:01.958892 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.958442    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:53:01.962040 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:01.962018    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2aaf48a6-openshift-default-59d7559c46-zc2x4"]
Jul 28 23:53:02.115523 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:02.115493    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" path="/var/lib/kubelet/pods/48c90f3b-a2be-4856-96f1-0c25b60d3dcf/volumes"
Jul 28 23:53:23.790828 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.790792    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"]
Jul 28 23:53:23.791466 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791446    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791467    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791483    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791491    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791512    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" containerName="payload-pre-processing"
Jul 28 23:53:23.791569 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" containerName="payload-pre-processing"
Jul 28 23:53:23.791813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791647    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791661    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c90f3b-a2be-4856-96f1-0c25b60d3dcf" containerName="istio-proxy"
Jul 28 23:53:23.791813 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.791674    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="550fcdfb-21c9-4e22-bcdf-ceec3f749273" containerName="payload-pre-processing"
Jul 28 23:53:23.794934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.794914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:23.798191 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.797986    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:53:23.801760 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.801733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"]
Jul 28 23:53:23.902774 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:23.902734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k\" (UID: \"a01cdc74-5397-4b4f-94cc-cc904186ff66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:24.003794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:24.003753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k\" (UID: \"a01cdc74-5397-4b4f-94cc-cc904186ff66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:24.004416 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:24.004394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k\" (UID: \"a01cdc74-5397-4b4f-94cc-cc904186ff66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:24.107877 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:24.107789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:24.458792 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:24.458758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"]
Jul 28 23:53:24.460292 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:53:24.460260    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01cdc74_5397_4b4f_94cc_cc904186ff66.slice/crio-2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e WatchSource:0}: Error finding container 2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e: Status 404 returned error can't find the container with id 2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e
Jul 28 23:53:25.067773 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:25.067738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k" event={"ID":"a01cdc74-5397-4b4f-94cc-cc904186ff66","Type":"ContainerStarted","Data":"9df280095ec4c386612f46ec2a57830a9ac29855d81f4934236f86b3a18bb7ba"}
Jul 28 23:53:25.068136 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:25.067799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k" event={"ID":"a01cdc74-5397-4b4f-94cc-cc904186ff66","Type":"ContainerStarted","Data":"2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e"}
Jul 28 23:53:25.103816 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:25.103759    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k" podStartSLOduration=2.103744199 podStartE2EDuration="2.103744199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:53:25.101686551 +0000 UTC m=+1905.628735880" watchObservedRunningTime="2026-07-28 23:53:25.103744199 +0000 UTC m=+1905.630793461"
Jul 28 23:53:26.075543 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:26.075505    2564 generic.go:356] "Generic (PLEG): container finished" podID="a01cdc74-5397-4b4f-94cc-cc904186ff66" containerID="9df280095ec4c386612f46ec2a57830a9ac29855d81f4934236f86b3a18bb7ba" exitCode=0
Jul 28 23:53:26.075943 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:26.075589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k" event={"ID":"a01cdc74-5397-4b4f-94cc-cc904186ff66","Type":"ContainerDied","Data":"9df280095ec4c386612f46ec2a57830a9ac29855d81f4934236f86b3a18bb7ba"}
Jul 28 23:53:27.227978 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:27.227946    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:27.342195 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:27.342103    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca\") pod \"a01cdc74-5397-4b4f-94cc-cc904186ff66\" (UID: \"a01cdc74-5397-4b4f-94cc-cc904186ff66\") "
Jul 28 23:53:27.342499 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:27.342476    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a01cdc74-5397-4b4f-94cc-cc904186ff66" (UID: "a01cdc74-5397-4b4f-94cc-cc904186ff66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:53:27.443334 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:27.443297    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a01cdc74-5397-4b4f-94cc-cc904186ff66-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:53:28.088057 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:28.088013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k" event={"ID":"a01cdc74-5397-4b4f-94cc-cc904186ff66","Type":"ContainerDied","Data":"2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e"}
Jul 28 23:53:28.088280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:28.088079    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2383c5adb004e8b113f1a703692e7322c897076739215d27bc0424134ce7301e"
Jul 28 23:53:28.088280 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:28.088027    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"
Jul 28 23:53:29.905661 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:29.905619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:30.083561 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:30.083517    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"]
Jul 28 23:53:30.085857 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:30.085837    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-765c35ee-7vk4k"]
Jul 28 23:53:30.115703 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:30.115672    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a01cdc74-5397-4b4f-94cc-cc904186ff66" path="/var/lib/kubelet/pods/a01cdc74-5397-4b4f-94cc-cc904186ff66/volumes"
Jul 28 23:53:31.151434 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:31.151400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:32.463869 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:32.463827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:38.091431 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:38.091396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:38.164914 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:38.164877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:38.236072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:38.236028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:38.336618 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:38.336579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:53:38.756622 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:53:38.756585    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:04.695488 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:04.695445    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:11.146781 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:11.146738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:11.254990 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:11.254947    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:11.350160 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:11.350126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:27.188941 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:27.188901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:33.084103 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:33.084008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:33.146957 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:33.146921    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:33.222698 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:33.222656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:33.322074 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:33.322045    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:43.792692 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:43.792657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:44.655338 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:44.655297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:50.868566 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:50.868526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:50.977658 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:50.977607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:51.074012 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:51.073972    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:54:51.171134 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:54:51.171037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:03.363657 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:03.363616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:08.921167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:08.921124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:09.024546 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:09.024510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:09.124258 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:09.124221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:10.746137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:10.746099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:16.870138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:16.870094    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:16.970254 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:16.970217    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:17.546768 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:17.546724    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:23.353954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:23.353898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:23.417561 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:23.417521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:23.590929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:23.590895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:23.889147 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:23.889112    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:24.162262 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:24.162168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:27.154911 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.154876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:27.261732 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.261695    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:27.287303 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.287270    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:55:27.287904 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.287882    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01cdc74-5397-4b4f-94cc-cc904186ff66" containerName="revoke-keys"
Jul 28 23:55:27.287904 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.287905    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01cdc74-5397-4b4f-94cc-cc904186ff66" containerName="revoke-keys"
Jul 28 23:55:27.288074 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.288009    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01cdc74-5397-4b4f-94cc-cc904186ff66" containerName="revoke-keys"
Jul 28 23:55:27.291406 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.291386    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.296512 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.296493    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-790b15df-serving-cert\""
Jul 28 23:55:27.297480 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.297459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 28 23:55:27.304088 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.304055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.304221 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.304094    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzcd6\" (UniqueName: \"kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.304221 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.304155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.328633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.328591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:55:27.394266 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.394226    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:27.405649 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.405568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.405649 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.405606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzcd6\" (UniqueName: \"kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.405905 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.405657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.405905 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:55:27.405744    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-serving-cert: secret "maas-api-e2e-mt-790b15df-serving-cert" not found
Jul 28 23:55:27.405905 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:55:27.405822    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls podName:f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a nodeName:}" failed. No retries permitted until 2026-07-28 23:55:27.905799464 +0000 UTC m=+2028.432848705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls") pod "maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" (UID: "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a") : secret "maas-api-e2e-mt-790b15df-serving-cert" not found
Jul 28 23:55:27.406395 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.406376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.431749 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.431716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzcd6\" (UniqueName: \"kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.601099 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.601062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:27.910237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.910198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.912880 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.912848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") pod \"maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:27.938748 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:27.938705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:28.028889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:28.028855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:28.205524 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:28.205432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:28.390046 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:28.390021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:55:28.392907 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:55:28.392875    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c4c7e0_2bea_4e2e_b877_d62d204a7f6a.slice/crio-1241bc7ba5783fe3019c9a11245d08eb5efa631a55c9737dca9420315e9fed96 WatchSource:0}: Error finding container 1241bc7ba5783fe3019c9a11245d08eb5efa631a55c9737dca9420315e9fed96: Status 404 returned error can't find the container with id 1241bc7ba5783fe3019c9a11245d08eb5efa631a55c9737dca9420315e9fed96
Jul 28 23:55:28.832244 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:28.832208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" event={"ID":"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a","Type":"ContainerStarted","Data":"1241bc7ba5783fe3019c9a11245d08eb5efa631a55c9737dca9420315e9fed96"}
Jul 28 23:55:29.840071 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:29.840028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" event={"ID":"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a","Type":"ContainerStarted","Data":"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"}
Jul 28 23:55:29.840479 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:29.840119    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:29.858021 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:29.857973    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" podStartSLOduration=2.4879769019999998 podStartE2EDuration="2.857958035s" totalImagesPullingTime="369.981133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:55:29.855244024 +0000 UTC m=+2030.382293286" watchObservedRunningTime="2026-07-28 23:55:29.857958035 +0000 UTC m=+2030.385007296"
Jul 28 23:55:35.852282 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:35.852248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:55:52.118084 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:52.118046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:52.772162 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:52.772118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:52.871178 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:52.871146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:53.210175 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:53.210131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:58.703037 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:58.702995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:58.760917 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:58.760880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:55:59.050180 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.050144    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"]
Jul 28 23:55:59.054072 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.054054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:55:59.056708 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.056645    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:55:59.061147 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.061114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"]
Jul 28 23:55:59.210579 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.210541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-790b15df-245bk\" (UID: \"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:55:59.311677 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.311571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-790b15df-245bk\" (UID: \"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:55:59.312214 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.312190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-790b15df-245bk\" (UID: \"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:55:59.366869 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.366835    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:55:59.510854 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:55:59.510817    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883a3ac8_aeff_4bc4_91c2_0d94bbd682ef.slice/crio-8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f WatchSource:0}: Error finding container 8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f: Status 404 returned error can't find the container with id 8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f
Jul 28 23:55:59.511526 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:55:59.511494    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"]
Jul 28 23:56:00.022203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:00.022104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk" event={"ID":"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef","Type":"ContainerStarted","Data":"7eecec8989261d3a0ccee1163323f5e8db42ce43837595930e9039f2d1b45a2a"}
Jul 28 23:56:00.022203 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:00.022167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk" event={"ID":"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef","Type":"ContainerStarted","Data":"8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f"}
Jul 28 23:56:00.036191 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:00.036140    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk" podStartSLOduration=1.036123434 podStartE2EDuration="1.036123434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:56:00.035580511 +0000 UTC m=+2060.562629779" watchObservedRunningTime="2026-07-28 23:56:00.036123434 +0000 UTC m=+2060.563172696"
Jul 28 23:56:01.030747 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:01.030706    2564 generic.go:356] "Generic (PLEG): container finished" podID="883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" containerID="7eecec8989261d3a0ccee1163323f5e8db42ce43837595930e9039f2d1b45a2a" exitCode=0
Jul 28 23:56:01.031193 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:01.030795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk" event={"ID":"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef","Type":"ContainerDied","Data":"7eecec8989261d3a0ccee1163323f5e8db42ce43837595930e9039f2d1b45a2a"}
Jul 28 23:56:02.182437 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:02.182369    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:56:02.342794 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:02.342700    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca\") pod \"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef\" (UID: \"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef\") "
Jul 28 23:56:02.343103 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:02.343082    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" (UID: "883a3ac8-aeff-4bc4-91c2-0d94bbd682ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:56:02.443601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:02.443563    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:56:03.049153 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:03.049121    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"
Jul 28 23:56:03.049153 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:03.049128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk" event={"ID":"883a3ac8-aeff-4bc4-91c2-0d94bbd682ef","Type":"ContainerDied","Data":"8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f"}
Jul 28 23:56:03.049398 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:03.049175    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ed2c4d1407d12751b0c5c43f814750f898eee0cb180a27a917c1dc407524a6f"
Jul 28 23:56:09.590811 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.590777    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:56:09.591301 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.591041    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" podUID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" containerName="maas-api" containerID="cri-o://2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194" gracePeriod=30
Jul 28 23:56:09.616066 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.616029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:09.682231 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.682192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:09.747328 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.747294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:09.777672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.777637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"]
Jul 28 23:56:09.780029 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.780006    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-790b15df-245bk"]
Jul 28 23:56:09.803655 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.803618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:09.846732 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.846661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:56:09.912252 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.912209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") pod \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") "
Jul 28 23:56:09.912496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.912269    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca\") pod \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") "
Jul 28 23:56:09.912496 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.912303    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzcd6\" (UniqueName: \"kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6\") pod \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\" (UID: \"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a\") "
Jul 28 23:56:09.912717 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.912692    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" (UID: "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:56:09.914782 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.914748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6" (OuterVolumeSpecName: "kube-api-access-bzcd6") pod "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" (UID: "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a"). InnerVolumeSpecName "kube-api-access-bzcd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:56:09.914913 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:09.914779    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" (UID: "f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:56:10.012992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.012956    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:56:10.012992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.012987    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:56:10.012992 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.012997    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzcd6\" (UniqueName: \"kubernetes.io/projected/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a-kube-api-access-bzcd6\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:56:10.091484 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.091445    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" containerID="2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194" exitCode=0
Jul 28 23:56:10.091633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.091493    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"
Jul 28 23:56:10.091633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.091533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" event={"ID":"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a","Type":"ContainerDied","Data":"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"}
Jul 28 23:56:10.091633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.091589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg" event={"ID":"f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a","Type":"ContainerDied","Data":"1241bc7ba5783fe3019c9a11245d08eb5efa631a55c9737dca9420315e9fed96"}
Jul 28 23:56:10.091633 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.091616    2564 scope.go:117] "RemoveContainer" containerID="2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"
Jul 28 23:56:10.102137 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.102117    2564 scope.go:117] "RemoveContainer" containerID="2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"
Jul 28 23:56:10.102426 ip-10-0-135-12 kubenswrapper[2564]: E0728 23:56:10.102406    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194\": container with ID starting with 2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194 not found: ID does not exist" containerID="2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"
Jul 28 23:56:10.102556 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.102435    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194"} err="failed to get container status \"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194\": rpc error: code = NotFound desc = could not find container \"2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194\": container with ID starting with 2d0b55b8b54a2e69825ba93fbfd56299375662b048673a7ad045042a6486d194 not found: ID does not exist"
Jul 28 23:56:10.118628 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.118598    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" path="/var/lib/kubelet/pods/883a3ac8-aeff-4bc4-91c2-0d94bbd682ef/volumes"
Jul 28 23:56:10.118873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.118859    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:56:10.118915 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.118886    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-790b15df-7bd59ccb64-bsfhg"]
Jul 28 23:56:10.146857 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:10.146829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:12.115672 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:12.115636    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" path="/var/lib/kubelet/pods/f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a/volumes"
Jul 28 23:56:16.130026 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.129988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.193098 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.193068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.389930 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.389839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.619321 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.619277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.755635 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.755542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.854563 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.854526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:16.951873 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:16.951841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:22.623018 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:22.622978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:22.724144 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:22.724093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:22.828642 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:22.828605    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:22.960262 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:22.960172    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:28.628706 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:28.628652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:28.730133 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:28.730089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:35.431300 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:35.431266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:35.735167 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:35.735075    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:36.117601 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:36.117569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:38.457881 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:38.457834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:38.557700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:38.557664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:38.620259 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:38.620224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:38.988798 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:38.988762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:39.148059 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:39.148022    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:39.246212 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:39.246119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:56:40.202996 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:40.202968    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:56:40.210076 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:40.210050    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:56:40.212237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:40.212213    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 28 23:56:40.218498 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:56:40.218480    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 28 23:57:03.428986 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:03.428936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:57:12.828946 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:12.828905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:57:37.673607 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.673571    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"]
Jul 28 23:57:37.674067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.673997    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" containerName="revoke-keys"
Jul 28 23:57:37.674067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.674009    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" containerName="revoke-keys"
Jul 28 23:57:37.674067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.674027    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" containerName="maas-api"
Jul 28 23:57:37.674067 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.674033    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" containerName="maas-api"
Jul 28 23:57:37.674197 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.674102    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="883a3ac8-aeff-4bc4-91c2-0d94bbd682ef" containerName="revoke-keys"
Jul 28 23:57:37.674197 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.674114    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3c4c7e0-2bea-4e2e-b877-d62d204a7f6a" containerName="maas-api"
Jul 28 23:57:37.677301 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.677284    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:37.679742 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.679710    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:57:37.683274 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.683251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"]
Jul 28 23:57:37.831299 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.831265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-21449885-gsr69\" (UID: \"7d70f1d2-ce54-4825-9045-a35bdbc84b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:37.932139 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.932049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-21449885-gsr69\" (UID: \"7d70f1d2-ce54-4825-9045-a35bdbc84b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:37.932730 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.932709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-21449885-gsr69\" (UID: \"7d70f1d2-ce54-4825-9045-a35bdbc84b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:37.990611 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:37.990573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:38.129129 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:38.129101    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"]
Jul 28 23:57:38.130513 ip-10-0-135-12 kubenswrapper[2564]: W0728 23:57:38.130489    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d70f1d2_ce54_4825_9045_a35bdbc84b99.slice/crio-8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07 WatchSource:0}: Error finding container 8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07: Status 404 returned error can't find the container with id 8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07
Jul 28 23:57:38.623331 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:38.623291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69" event={"ID":"7d70f1d2-ce54-4825-9045-a35bdbc84b99","Type":"ContainerStarted","Data":"df719e411aca1317ebef6f27a6fabe99442dce26b3ef5fd61ad65aa4f4a80ee2"}
Jul 28 23:57:38.623541 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:38.623370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69" event={"ID":"7d70f1d2-ce54-4825-9045-a35bdbc84b99","Type":"ContainerStarted","Data":"8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07"}
Jul 28 23:57:38.639166 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:38.639068    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69" podStartSLOduration=1.639052011 podStartE2EDuration="1.639052011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:57:38.636278516 +0000 UTC m=+2159.163327779" watchObservedRunningTime="2026-07-28 23:57:38.639052011 +0000 UTC m=+2159.166101294"
Jul 28 23:57:39.636721 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:39.636683    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d70f1d2-ce54-4825-9045-a35bdbc84b99" containerID="df719e411aca1317ebef6f27a6fabe99442dce26b3ef5fd61ad65aa4f4a80ee2" exitCode=0
Jul 28 23:57:39.637125 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:39.636771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69" event={"ID":"7d70f1d2-ce54-4825-9045-a35bdbc84b99","Type":"ContainerDied","Data":"df719e411aca1317ebef6f27a6fabe99442dce26b3ef5fd61ad65aa4f4a80ee2"}
Jul 28 23:57:40.300828 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:40.300797    2564 scope.go:117] "RemoveContainer" containerID="7e8caa98c29670c98d9aa5b59fc847386a64e6d3f9e6abdda0f27d8658a3eaae"
Jul 28 23:57:40.309824 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:40.309802    2564 scope.go:117] "RemoveContainer" containerID="8de5013878ca013508df5511c430427844f85c6a558d2febc1ee70838d6327de"
Jul 28 23:57:40.785512 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:40.785488    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:40.960121 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:40.960021    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca\") pod \"7d70f1d2-ce54-4825-9045-a35bdbc84b99\" (UID: \"7d70f1d2-ce54-4825-9045-a35bdbc84b99\") "
Jul 28 23:57:40.960398 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:40.960376    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d70f1d2-ce54-4825-9045-a35bdbc84b99" (UID: "7d70f1d2-ce54-4825-9045-a35bdbc84b99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:57:41.060862 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:41.060831    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d70f1d2-ce54-4825-9045-a35bdbc84b99-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 28 23:57:41.651118 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:41.651086    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"
Jul 28 23:57:41.651303 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:41.651117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69" event={"ID":"7d70f1d2-ce54-4825-9045-a35bdbc84b99","Type":"ContainerDied","Data":"8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07"}
Jul 28 23:57:41.651303 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:41.651166    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8eb5d1a256e23ed3d2fdc3f66d3ffcca7a25f6cbd1212a223b205f8b96033d07"
Jul 28 23:57:53.122046 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:53.122002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:57:53.221390 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:53.221328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:57:53.270674 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:53.270637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"]
Jul 28 23:57:53.272902 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:53.272880    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-21449885-gsr69"]
Jul 28 23:57:53.274715 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:53.274690    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:57:54.115315 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:54.115274    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d70f1d2-ce54-4825-9045-a35bdbc84b99" path="/var/lib/kubelet/pods/7d70f1d2-ce54-4825-9045-a35bdbc84b99/volumes"
Jul 28 23:57:56.306929 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:57:56.306878    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:01.386200 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:01.386159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:30.289084 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:30.289042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:37.605597 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:37.605553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:39.047895 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:39.047857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:40.338565 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:40.338538    2564 scope.go:117] "RemoveContainer" containerID="053fc3ec467bcffbf70bfd2f74aad70730d6c64dc4365018ffffdc3d99fecd7c"
Jul 28 23:58:43.768995 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:43.768955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:57.701859 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:57.701813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:57.766629 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:57.766582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:57.987557 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:57.987451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:58.230036 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:58.229989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:59.240236 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:59.240199    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:59.339784 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:59.339748    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:58:59.439720 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:58:59.439675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:00.145153 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:00.145111    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:00.247812 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:00.247777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:00.345926 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:00.345876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:01.652596 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:01.652548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:07.287086 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:07.287005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:07.386592 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:07.386543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:07.938900 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:07.938865    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:13.756154 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:13.756108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:13.877326 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:13.877280    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:13.969934 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:13.969885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:24.454479 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:24.454439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:24.785814 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:24.785704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:37.432888 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:37.432821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:40.361237 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:40.361202    2564 scope.go:117] "RemoveContainer" containerID="9df280095ec4c386612f46ec2a57830a9ac29855d81f4934236f86b3a18bb7ba"
Jul 28 23:59:45.734858 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:45.734815    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:52.210954 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:52.210914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:52.311889 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:52.311854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:52.564362 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:52.564296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:52.866064 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:52.865976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:52.962396 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:52.962325    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:55.409025 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:55.408979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:55.509138 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:55.509105    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:55.656474 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:55.656429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:55.752667 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:55.752572    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:56.176745 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:56.176703    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 28 23:59:56.276700 ip-10-0-135-12 kubenswrapper[2564]: I0728 23:59:56.276660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:00.128942 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.128900    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"]
Jul 29 00:00:00.129453 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.129435    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d70f1d2-ce54-4825-9045-a35bdbc84b99" containerName="revoke-keys"
Jul 29 00:00:00.129536 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.129456    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d70f1d2-ce54-4825-9045-a35bdbc84b99" containerName="revoke-keys"
Jul 29 00:00:00.129536 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.129534    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d70f1d2-ce54-4825-9045-a35bdbc84b99" containerName="revoke-keys"
Jul 29 00:00:00.132931 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.132904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"
Jul 29 00:00:00.135386 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.135358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:00:00.135850 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.135819    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:00.140533 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.140505    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29754720-xswqb"]
Jul 29 00:00:00.141231 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.141209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"
Jul 29 00:00:00.144610 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.144586    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"]
Jul 29 00:00:00.144739 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.144692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.146303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.146282    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29754720-xswqb"]
Jul 29 00:00:00.147157 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.147135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Jul 29 00:00:00.147560 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.147205    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-rcb98\""
Jul 29 00:00:00.155382 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.155356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:00.166363 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.166327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"
Jul 29 00:00:00.175664 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.175635    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"
Jul 29 00:00:00.203800 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.203414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgcj\" (UniqueName: \"kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.203800 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.203487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.304955 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.304884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgcj\" (UniqueName: \"kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.305150 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.304961    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.305944 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.305912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.316094 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.315749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgcj\" (UniqueName: \"kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj\") pod \"image-pruner-29754720-xswqb\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") " pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.348533 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.348483    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"]
Jul 29 00:00:00.350589 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:00:00.350551    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11608e0_22d6_4e63_85a3_03e989de51b6.slice/crio-dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3 WatchSource:0}: Error finding container dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3: Status 404 returned error can't find the container with id dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3
Jul 29 00:00:00.385153 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.384736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:00.491862 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.491823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerStarted","Data":"d31251407580ce7f8be7e3f81e480ba5d3c4805c33b5c180dea49fd4cbb7745d"}
Jul 29 00:00:00.493165 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.493135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w" event={"ID":"b11608e0-22d6-4e63-85a3-03e989de51b6","Type":"ContainerStarted","Data":"dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3"}
Jul 29 00:00:00.513099 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.513061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:00.662539 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:00.662496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29754720-xswqb"]
Jul 29 00:00:00.664355 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:00:00.664308    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdcae63_d249_4050_b85f_7d69f565f896.slice/crio-e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3 WatchSource:0}: Error finding container e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3: Status 404 returned error can't find the container with id e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3
Jul 29 00:00:01.500532 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.500497    2564 generic.go:356] "Generic (PLEG): container finished" podID="b11608e0-22d6-4e63-85a3-03e989de51b6" containerID="3f4f4146ef420160cdcb3ef3aac4210a67f3db8f47414ed9b06232abc8398853" exitCode=0
Jul 29 00:00:01.500932 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.500585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w" event={"ID":"b11608e0-22d6-4e63-85a3-03e989de51b6","Type":"ContainerDied","Data":"3f4f4146ef420160cdcb3ef3aac4210a67f3db8f47414ed9b06232abc8398853"}
Jul 29 00:00:01.502068 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.502043    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerID="cbbd47e7e0a080b0440cd781e20b318c2a5b6cf9e4fd6e8a883b47f7d6fdc5e8" exitCode=7
Jul 29 00:00:01.502187 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.502120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerDied","Data":"cbbd47e7e0a080b0440cd781e20b318c2a5b6cf9e4fd6e8a883b47f7d6fdc5e8"}
Jul 29 00:00:01.502338 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.502241    2564 scope.go:117] "RemoveContainer" containerID="cbbd47e7e0a080b0440cd781e20b318c2a5b6cf9e4fd6e8a883b47f7d6fdc5e8"
Jul 29 00:00:01.503980 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.503960    2564 generic.go:356] "Generic (PLEG): container finished" podID="6fdcae63-d249-4050-b85f-7d69f565f896" containerID="e1599deefcf7717ea1247fa7a082ab9bacf874dfd2074def5b11612684ddecbd" exitCode=0
Jul 29 00:00:01.504072 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.504029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29754720-xswqb" event={"ID":"6fdcae63-d249-4050-b85f-7d69f565f896","Type":"ContainerDied","Data":"e1599deefcf7717ea1247fa7a082ab9bacf874dfd2074def5b11612684ddecbd"}
Jul 29 00:00:01.504072 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:01.504059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29754720-xswqb" event={"ID":"6fdcae63-d249-4050-b85f-7d69f565f896","Type":"ContainerStarted","Data":"e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3"}
Jul 29 00:00:02.511743 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.511647    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerID="92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6" exitCode=7
Jul 29 00:00:02.512172 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.511736    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerDied","Data":"92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6"}
Jul 29 00:00:02.512172 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.511800    2564 scope.go:117] "RemoveContainer" containerID="cbbd47e7e0a080b0440cd781e20b318c2a5b6cf9e4fd6e8a883b47f7d6fdc5e8"
Jul 29 00:00:02.512172 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.512099    2564 scope.go:117] "RemoveContainer" containerID="92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6"
Jul 29 00:00:02.512887 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:00:02.512416    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b_odh-ai-gateway-infra(3c7d1dd8-1c3f-41cf-811e-052a52a07071)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071"
Jul 29 00:00:02.654229 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.654203    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"
Jul 29 00:00:02.699403 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.699372    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:02.830449 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.830410    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cgcj\" (UniqueName: \"kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj\") pod \"6fdcae63-d249-4050-b85f-7d69f565f896\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") "
Jul 29 00:00:02.830635 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.830467    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca\") pod \"6fdcae63-d249-4050-b85f-7d69f565f896\" (UID: \"6fdcae63-d249-4050-b85f-7d69f565f896\") "
Jul 29 00:00:02.830856 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.830832    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca" (OuterVolumeSpecName: "serviceca") pod "6fdcae63-d249-4050-b85f-7d69f565f896" (UID: "6fdcae63-d249-4050-b85f-7d69f565f896"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:00:02.832620 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.832598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj" (OuterVolumeSpecName: "kube-api-access-7cgcj") pod "6fdcae63-d249-4050-b85f-7d69f565f896" (UID: "6fdcae63-d249-4050-b85f-7d69f565f896"). InnerVolumeSpecName "kube-api-access-7cgcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:00:02.931918 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.931869    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cgcj\" (UniqueName: \"kubernetes.io/projected/6fdcae63-d249-4050-b85f-7d69f565f896-kube-api-access-7cgcj\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:00:02.931918 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:02.931909    2564 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fdcae63-d249-4050-b85f-7d69f565f896-serviceca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:00:03.521658 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.521560    2564 scope.go:117] "RemoveContainer" containerID="92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6"
Jul 29 00:00:03.522112 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:00:03.521882    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b_odh-ai-gateway-infra(3c7d1dd8-1c3f-41cf-811e-052a52a07071)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071"
Jul 29 00:00:03.523045 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.523019    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29754720-xswqb"
Jul 29 00:00:03.523174 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.523044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29754720-xswqb" event={"ID":"6fdcae63-d249-4050-b85f-7d69f565f896","Type":"ContainerDied","Data":"e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3"}
Jul 29 00:00:03.523174 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.523095    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0e2467a20a3e33317ace9bf4bcd3e4730dfbf7396fe2a2840da3450a69c66d3"
Jul 29 00:00:03.524638 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.524618    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w"
Jul 29 00:00:03.524638 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.524631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754720-mkr6w" event={"ID":"b11608e0-22d6-4e63-85a3-03e989de51b6","Type":"ContainerDied","Data":"dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3"}
Jul 29 00:00:03.524836 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:03.524667    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dad4e84401561f4000157b89cd013198ff3d36d7222173ffb251a22a345224f3"
Jul 29 00:00:06.573010 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:06.572973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:17.109119 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:17.109086    2564 scope.go:117] "RemoveContainer" containerID="92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6"
Jul 29 00:00:17.610785 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:17.610747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerStarted","Data":"139262acb264f72d0739d6af7756b1e236a90be8f5a75d6b7be12c4934f53dcd"}
Jul 29 00:00:17.627205 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:17.627101    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" podStartSLOduration=17.627085141 podStartE2EDuration="17.627085141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:00:17.624596999 +0000 UTC m=+2318.151646263" watchObservedRunningTime="2026-07-29 00:00:17.627085141 +0000 UTC m=+2318.154134404"
Jul 29 00:00:18.618619 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:18.618579    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerID="139262acb264f72d0739d6af7756b1e236a90be8f5a75d6b7be12c4934f53dcd" exitCode=0
Jul 29 00:00:18.619027 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:18.618649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerDied","Data":"139262acb264f72d0739d6af7756b1e236a90be8f5a75d6b7be12c4934f53dcd"}
Jul 29 00:00:18.619027 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:18.618711    2564 scope.go:117] "RemoveContainer" containerID="92a6e312a9791254f4640b8a9b73e72e5969b8bcb4e341d7cfde85b767826dd6"
Jul 29 00:00:18.635137 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:18.635107    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:19.769611 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:19.769585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"
Jul 29 00:00:20.633496 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.633467    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"
Jul 29 00:00:20.633704 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.633464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b" event={"ID":"3c7d1dd8-1c3f-41cf-811e-052a52a07071","Type":"ContainerDied","Data":"d31251407580ce7f8be7e3f81e480ba5d3c4805c33b5c180dea49fd4cbb7745d"}
Jul 29 00:00:20.633704 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.633635    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d31251407580ce7f8be7e3f81e480ba5d3c4805c33b5c180dea49fd4cbb7745d"
Jul 29 00:00:20.664780 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.664744    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:20.670562 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.670533    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-c418f4-29754720-mr57b"]
Jul 29 00:00:20.673461 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.673430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:20.770316 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:20.770273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:21.011724 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:21.011622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:22.115795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:22.115764    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" path="/var/lib/kubelet/pods/3c7d1dd8-1c3f-41cf-811e-052a52a07071/volumes"
Jul 29 00:00:22.426367 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:22.426256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:23.517225 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.517180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:23.596240 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.596200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:23.599054 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599023    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:00:23.599566 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599545    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fdcae63-d249-4050-b85f-7d69f565f896" containerName="image-pruner"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599568    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fdcae63-d249-4050-b85f-7d69f565f896" containerName="image-pruner"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599582    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599591    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599622    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599631    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599653    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.599673 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599680    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11608e0-22d6-4e63-85a3-03e989de51b6" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599688    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11608e0-22d6-4e63-85a3-03e989de51b6" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599788    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599803    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11608e0-22d6-4e63-85a3-03e989de51b6" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599819    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:00:23.600035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.599830    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fdcae63-d249-4050-b85f-7d69f565f896" containerName="image-pruner"
Jul 29 00:00:23.603447 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.603419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.605956 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.605930    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b178a9-serving-cert\""
Jul 29 00:00:23.605956 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.605959    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 29 00:00:23.612597 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.611999    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:00:23.629021 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.628987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jblnr\" (UniqueName: \"kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.629207 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.629130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.629207 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.629162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.662438 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.662399    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:23.729855 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.729816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jblnr\" (UniqueName: \"kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.730064 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.729901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.730064 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.729921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.730064 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:00:23.730039    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-serving-cert: secret "maas-api-e2e-inf-b-b178a9-serving-cert" not found
Jul 29 00:00:23.730237 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:00:23.730116    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls podName:04846cdd-0bf1-4aaf-8b44-0af15a3348aa nodeName:}" failed. No retries permitted until 2026-07-29 00:00:24.230095459 +0000 UTC m=+2324.757144699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls") pod "maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" (UID: "04846cdd-0bf1-4aaf-8b44-0af15a3348aa") : secret "maas-api-e2e-inf-b-b178a9-serving-cert" not found
Jul 29 00:00:23.730554 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.730536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.741412 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.741382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jblnr\" (UniqueName: \"kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:23.788248 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:23.788151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:24.101187 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.101144    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:24.235407 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.235368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:24.238139 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.238107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") pod \"maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:24.415951 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.415857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:00:24.517508 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.517468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:24.665209 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.665166    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:00:24.672578 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:00:24.672546    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04846cdd_0bf1_4aaf_8b44_0af15a3348aa.slice/crio-be96c71e2dc697f4026798029a0a2e67abf2f8f65b5fc841c7710a5e5526712f WatchSource:0}: Error finding container be96c71e2dc697f4026798029a0a2e67abf2f8f65b5fc841c7710a5e5526712f: Status 404 returned error can't find the container with id be96c71e2dc697f4026798029a0a2e67abf2f8f65b5fc841c7710a5e5526712f
Jul 29 00:00:24.674188 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:24.674166    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:00:25.682264 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:25.682161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" event={"ID":"04846cdd-0bf1-4aaf-8b44-0af15a3348aa","Type":"ContainerStarted","Data":"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"}
Jul 29 00:00:25.682264 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:25.682228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" event={"ID":"04846cdd-0bf1-4aaf-8b44-0af15a3348aa","Type":"ContainerStarted","Data":"be96c71e2dc697f4026798029a0a2e67abf2f8f65b5fc841c7710a5e5526712f"}
Jul 29 00:00:25.682733 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:25.682271    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:25.698521 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:25.698470    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" podStartSLOduration=1.967607211 podStartE2EDuration="2.698451901s" totalImagesPullingTime="730.84469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:00:25.697888739 +0000 UTC m=+2326.224938014" watchObservedRunningTime="2026-07-29 00:00:25.698451901 +0000 UTC m=+2326.225501163"
Jul 29 00:00:31.694283 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:31.694245    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:00:49.317095 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:00:49.317052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:01:03.106549 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:03.106496    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:01:33.706138 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:33.706101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:01:40.248564 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:40.248537    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:01:40.254847 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:40.254819    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:01:40.261702 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:40.261676    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:01:40.269049 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:01:40.269021    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:02:13.144934 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:13.144899    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:14.052833 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.052792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:14.184300 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.184263    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"]
Jul 29 00:02:14.184835 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.184818    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7d1dd8-1c3f-41cf-811e-052a52a07071" containerName="cleanup"
Jul 29 00:02:14.188098 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.188081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:14.190852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.190760    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:02:14.194622 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.194599    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"]
Jul 29 00:02:14.287582 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.287539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4\" (UID: \"9c639a84-9c3d-453e-a727-0996a0813fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:14.388975 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.388866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4\" (UID: \"9c639a84-9c3d-453e-a727-0996a0813fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:14.389503 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.389483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4\" (UID: \"9c639a84-9c3d-453e-a727-0996a0813fcd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:14.501624 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.501589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:14.848372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:14.848327    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"]
Jul 29 00:02:14.849934 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:02:14.849903    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c639a84_9c3d_453e_a727_0996a0813fcd.slice/crio-c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53 WatchSource:0}: Error finding container c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53: Status 404 returned error can't find the container with id c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53
Jul 29 00:02:15.345912 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:15.345868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4" event={"ID":"9c639a84-9c3d-453e-a727-0996a0813fcd","Type":"ContainerStarted","Data":"54eb5ec55322190ff9423b528d0b1b7aedc35373b3f7ab7c42abf5e515c47b1e"}
Jul 29 00:02:15.346297 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:15.345933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4" event={"ID":"9c639a84-9c3d-453e-a727-0996a0813fcd","Type":"ContainerStarted","Data":"c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53"}
Jul 29 00:02:15.363194 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:15.363141    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4" podStartSLOduration=1.363124748 podStartE2EDuration="1.363124748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:02:15.360050071 +0000 UTC m=+2435.887099337" watchObservedRunningTime="2026-07-29 00:02:15.363124748 +0000 UTC m=+2435.890174011"
Jul 29 00:02:16.353260 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:16.353223    2564 generic.go:356] "Generic (PLEG): container finished" podID="9c639a84-9c3d-453e-a727-0996a0813fcd" containerID="54eb5ec55322190ff9423b528d0b1b7aedc35373b3f7ab7c42abf5e515c47b1e" exitCode=0
Jul 29 00:02:16.353751 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:16.353319    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4" event={"ID":"9c639a84-9c3d-453e-a727-0996a0813fcd","Type":"ContainerDied","Data":"54eb5ec55322190ff9423b528d0b1b7aedc35373b3f7ab7c42abf5e515c47b1e"}
Jul 29 00:02:16.683612 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:16.683508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:17.503204 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:17.503178    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:17.619520 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:17.619417    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca\") pod \"9c639a84-9c3d-453e-a727-0996a0813fcd\" (UID: \"9c639a84-9c3d-453e-a727-0996a0813fcd\") "
Jul 29 00:02:17.619838 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:17.619815    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c639a84-9c3d-453e-a727-0996a0813fcd" (UID: "9c639a84-9c3d-453e-a727-0996a0813fcd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:17.721120 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:17.721081    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c639a84-9c3d-453e-a727-0996a0813fcd-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:02:18.367135 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:18.367098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4" event={"ID":"9c639a84-9c3d-453e-a727-0996a0813fcd","Type":"ContainerDied","Data":"c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53"}
Jul 29 00:02:18.367135 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:18.367131    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"
Jul 29 00:02:18.367388 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:18.367149    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8ea4af238a47ad50ef3aa0994dde9359f01ccf76d44957e0cb4e35b36609a53"
Jul 29 00:02:24.617977 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.617939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:24.687402 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.687364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:24.789251 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.789204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:24.906799 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.906705    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"]
Jul 29 00:02:24.910062 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.910029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:24.910279 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:24.910261    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c418f4-5f4g4"]
Jul 29 00:02:25.263013 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:25.262922    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:26.116311 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:26.116278    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c639a84-9c3d-453e-a727-0996a0813fcd" path="/var/lib/kubelet/pods/9c639a84-9c3d-453e-a727-0996a0813fcd/volumes"
Jul 29 00:02:31.013854 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.013812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:31.112477 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.112431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:31.257570 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.257526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:31.488609 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.488569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:31.593307 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.593270    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"]
Jul 29 00:02:31.593780 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.593766    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c639a84-9c3d-453e-a727-0996a0813fcd" containerName="revoke-keys"
Jul 29 00:02:31.593823 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.593782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c639a84-9c3d-453e-a727-0996a0813fcd" containerName="revoke-keys"
Jul 29 00:02:31.593876 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.593866    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c639a84-9c3d-453e-a727-0996a0813fcd" containerName="revoke-keys"
Jul 29 00:02:31.597300 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.597283    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:31.599822 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.599798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:02:31.603826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.603801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"]
Jul 29 00:02:31.757318 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.757226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5\" (UID: \"9d8d8487-f445-44b9-b24d-57eee5c8de86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:31.858234 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.858199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5\" (UID: \"9d8d8487-f445-44b9-b24d-57eee5c8de86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:31.858865 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.858845    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5\" (UID: \"9d8d8487-f445-44b9-b24d-57eee5c8de86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:31.912030 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:31.911994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:32.051705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:32.051676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"]
Jul 29 00:02:32.052915 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:02:32.052891    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8d8487_f445_44b9_b24d_57eee5c8de86.slice/crio-26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026 WatchSource:0}: Error finding container 26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026: Status 404 returned error can't find the container with id 26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026
Jul 29 00:02:32.450252 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:32.450214    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5" event={"ID":"9d8d8487-f445-44b9-b24d-57eee5c8de86","Type":"ContainerStarted","Data":"5e5437e8df515b652e9e2dcd6d07c30418b832ee36934a2986370d4e6f416256"}
Jul 29 00:02:32.450426 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:32.450278    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5" event={"ID":"9d8d8487-f445-44b9-b24d-57eee5c8de86","Type":"ContainerStarted","Data":"26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026"}
Jul 29 00:02:32.467292 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:32.467245    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5" podStartSLOduration=1.46722903 podStartE2EDuration="1.46722903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:02:32.465053019 +0000 UTC m=+2452.992102283" watchObservedRunningTime="2026-07-29 00:02:32.46722903 +0000 UTC m=+2452.994278297"
Jul 29 00:02:33.457873 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:33.457839    2564 generic.go:356] "Generic (PLEG): container finished" podID="9d8d8487-f445-44b9-b24d-57eee5c8de86" containerID="5e5437e8df515b652e9e2dcd6d07c30418b832ee36934a2986370d4e6f416256" exitCode=0
Jul 29 00:02:33.458267 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:33.457928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5" event={"ID":"9d8d8487-f445-44b9-b24d-57eee5c8de86","Type":"ContainerDied","Data":"5e5437e8df515b652e9e2dcd6d07c30418b832ee36934a2986370d4e6f416256"}
Jul 29 00:02:34.602011 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:34.601980    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:34.788608 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:34.788523    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca\") pod \"9d8d8487-f445-44b9-b24d-57eee5c8de86\" (UID: \"9d8d8487-f445-44b9-b24d-57eee5c8de86\") "
Jul 29 00:02:34.788936 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:34.788916    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d8d8487-f445-44b9-b24d-57eee5c8de86" (UID: "9d8d8487-f445-44b9-b24d-57eee5c8de86"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:34.890013 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:34.889976    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d8d8487-f445-44b9-b24d-57eee5c8de86-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:02:35.472915 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:35.472881    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"
Jul 29 00:02:35.473115 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:35.472913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5" event={"ID":"9d8d8487-f445-44b9-b24d-57eee5c8de86","Type":"ContainerDied","Data":"26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026"}
Jul 29 00:02:35.473115 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:35.472979    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26aeeaf530a085a2051cc1c9f9c1d6f8be4f3429d39946359b26d3a49346c026"
Jul 29 00:02:40.402099 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:40.402059    2564 scope.go:117] "RemoveContainer" containerID="7eecec8989261d3a0ccee1163323f5e8db42ce43837595930e9039f2d1b45a2a"
Jul 29 00:02:41.993259 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:41.993205    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:02:41.993772 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:41.993525    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" podUID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" containerName="maas-api" containerID="cri-o://f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37" gracePeriod=30
Jul 29 00:02:42.026175 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.026147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:42.123746 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.123714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:42.176554 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.176523    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"]
Jul 29 00:02:42.182733 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.182702    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b178a9-mgqx5"]
Jul 29 00:02:42.261508 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.261478    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:02:42.351255 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.351221    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") pod \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") "
Jul 29 00:02:42.351473 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.351334    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca\") pod \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") "
Jul 29 00:02:42.351473 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.351382    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jblnr\" (UniqueName: \"kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr\") pod \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\" (UID: \"04846cdd-0bf1-4aaf-8b44-0af15a3348aa\") "
Jul 29 00:02:42.351690 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.351669    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04846cdd-0bf1-4aaf-8b44-0af15a3348aa" (UID: "04846cdd-0bf1-4aaf-8b44-0af15a3348aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:42.353490 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.353459    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04846cdd-0bf1-4aaf-8b44-0af15a3348aa" (UID: "04846cdd-0bf1-4aaf-8b44-0af15a3348aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:02:42.353490 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.353459    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr" (OuterVolumeSpecName: "kube-api-access-jblnr") pod "04846cdd-0bf1-4aaf-8b44-0af15a3348aa" (UID: "04846cdd-0bf1-4aaf-8b44-0af15a3348aa"). InnerVolumeSpecName "kube-api-access-jblnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:42.452270 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.452238    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:02:42.452270 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.452268    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:02:42.452270 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.452278    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jblnr\" (UniqueName: \"kubernetes.io/projected/04846cdd-0bf1-4aaf-8b44-0af15a3348aa-kube-api-access-jblnr\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:02:42.522122 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.522014    2564 generic.go:356] "Generic (PLEG): container finished" podID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" containerID="f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37" exitCode=0
Jul 29 00:02:42.522122 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.522066    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"
Jul 29 00:02:42.522122 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.522105    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" event={"ID":"04846cdd-0bf1-4aaf-8b44-0af15a3348aa","Type":"ContainerDied","Data":"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"}
Jul 29 00:02:42.522424 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.522165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn" event={"ID":"04846cdd-0bf1-4aaf-8b44-0af15a3348aa","Type":"ContainerDied","Data":"be96c71e2dc697f4026798029a0a2e67abf2f8f65b5fc841c7710a5e5526712f"}
Jul 29 00:02:42.522424 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.522209    2564 scope.go:117] "RemoveContainer" containerID="f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"
Jul 29 00:02:42.532649 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.532628    2564 scope.go:117] "RemoveContainer" containerID="f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"
Jul 29 00:02:42.532888 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:02:42.532866    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37\": container with ID starting with f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37 not found: ID does not exist" containerID="f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"
Jul 29 00:02:42.532961 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.532900    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37"} err="failed to get container status \"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37\": rpc error: code = NotFound desc = could not find container \"f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37\": container with ID starting with f2fba16280ed928d5f467401e27f7324455df9b90c269ac4725c63dc95ae7e37 not found: ID does not exist"
Jul 29 00:02:42.543762 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.543737    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:02:42.547163 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.547141    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b178a9-5f6c564f49-btdcn"]
Jul 29 00:02:42.558869 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.558831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:42.611629 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:42.611592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:44.115814 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:44.115776    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" path="/var/lib/kubelet/pods/04846cdd-0bf1-4aaf-8b44-0af15a3348aa/volumes"
Jul 29 00:02:44.116295 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:44.116271    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8d8487-f445-44b9-b24d-57eee5c8de86" path="/var/lib/kubelet/pods/9d8d8487-f445-44b9-b24d-57eee5c8de86/volumes"
Jul 29 00:02:48.430239 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:48.430196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:48.498177 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:48.498140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:48.681571 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:48.681484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:48.899772 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:48.899740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:49.507070 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507031    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:02:49.507522 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507504    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" containerName="maas-api"
Jul 29 00:02:49.507597 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507525    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" containerName="maas-api"
Jul 29 00:02:49.507597 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507553    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8d8487-f445-44b9-b24d-57eee5c8de86" containerName="revoke-keys"
Jul 29 00:02:49.507597 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507559    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8d8487-f445-44b9-b24d-57eee5c8de86" containerName="revoke-keys"
Jul 29 00:02:49.507696 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507647    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8d8487-f445-44b9-b24d-57eee5c8de86" containerName="revoke-keys"
Jul 29 00:02:49.507696 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.507660    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04846cdd-0bf1-4aaf-8b44-0af15a3348aa" containerName="maas-api"
Jul 29 00:02:49.510938 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.510923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.513274 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.513256    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-519e1e-openshift-default-dockercfg-r82wq\""
Jul 29 00:02:49.521093 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.521072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:02:49.612758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fbzz\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.612922 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612775    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.612922 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.612982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.613000    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613170 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.613020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.613170 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.613045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714173 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714136    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714173 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fbzz\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714715 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714715 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714715 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714715 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714667    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.714971 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.715103 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.714983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.715276 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.715255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.716806 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.716783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.717032 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.717009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.722181 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.722157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.722605 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.722586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fbzz\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz\") pod \"e2e-api-a-519e1e-openshift-default-548b844756-446z7\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") " pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.822411 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.822381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:49.982748 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.982468    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:49.984824 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:02:49.984788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0853acaf_0076_4b85_913f_1a5bcf257f23.slice/crio-67bb98d266b0f47f2aaf50cf5279aec2d15e9ba42641e795f7dc08d22b842a97 WatchSource:0}: Error finding container 67bb98d266b0f47f2aaf50cf5279aec2d15e9ba42641e795f7dc08d22b842a97: Status 404 returned error can't find the container with id 67bb98d266b0f47f2aaf50cf5279aec2d15e9ba42641e795f7dc08d22b842a97
Jul 29 00:02:49.984824 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.984802    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:02:49.987127 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.987090    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:02:49.987219 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.987172    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:02:49.987219 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:49.987210    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:02:50.040112 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:50.040078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:50.573439 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:50.573390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerStarted","Data":"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"}
Jul 29 00:02:50.573825 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:50.573499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerStarted","Data":"67bb98d266b0f47f2aaf50cf5279aec2d15e9ba42641e795f7dc08d22b842a97"}
Jul 29 00:02:50.594488 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:50.594426    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podStartSLOduration=1.5944081940000001 podStartE2EDuration="1.594408194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:02:50.592665901 +0000 UTC m=+2471.119715165" watchObservedRunningTime="2026-07-29 00:02:50.594408194 +0000 UTC m=+2471.121457459"
Jul 29 00:02:50.823044 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:50.823006    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:51.827976 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:51.827944    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:52.499153 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:52.499118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:52.585918 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:52.585888    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:52.586870 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:52.586849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:02:52.601375 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:52.601335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:52.722611 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:52.722561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:53.019081 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:53.019043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:53.332954 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:53.332922    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:02:53.398467 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:02:53.398434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:07.587478 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.587429    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 29 00:03:07.587950 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.587515    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 29 00:03:07.677901 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.677865    2564 generic.go:356] "Generic (PLEG): container finished" podID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerID="e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b" exitCode=0
Jul 29 00:03:07.678110 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.677943    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerDied","Data":"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"}
Jul 29 00:03:07.678438 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.678419    2564 scope.go:117] "RemoveContainer" containerID="e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"
Jul 29 00:03:07.679509 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.679485    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:07.679575 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.679553    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:07.679633 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:07.679582    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:08.690615 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:08.690575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerStarted","Data":"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"}
Jul 29 00:03:08.823413 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:08.823377    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:03:08.823679 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:08.823655    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 29 00:03:08.823729 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:08.823709    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 29 00:03:09.822825 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:09.822787    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 29 00:03:09.823332 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:09.822853    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 29 00:03:10.823523 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:10.823489    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 29 00:03:10.823931 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:10.823552    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 29 00:03:11.823625 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:11.823590    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 29 00:03:11.824061 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:11.823673    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 29 00:03:12.827004 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:12.826974    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:03:12.827444 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:12.827310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:03:12.828182 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:12.828165    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:03:17.514109 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:17.514074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:17.964184 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:17.964146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:18.257546 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:18.257446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:18.463081 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:18.463035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:21.317872 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.317830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:21.402072 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.402032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:21.407785 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.407752    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:03:21.411708 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.411688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.414708 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.414683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-48f20e-serving-cert\""
Jul 29 00:03:21.414866 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.414849    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 29 00:03:21.424558 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.424533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:03:21.468117 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.468081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:21.551625 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.551587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlt6s\" (UniqueName: \"kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.551799 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.551718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.551799 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.551757    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.616917 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.616824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:21.652563 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.652530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt6s\" (UniqueName: \"kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.652739 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.652644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.652739 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.652687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.653453 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.653429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.655154 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.655131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.671430 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.671399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlt6s\" (UniqueName: \"kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s\") pod \"maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.725089 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.725046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:21.884395 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:21.884361    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:03:21.885241 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:03:21.885205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a89eb3e_9a2d_487e_a1e4_7f7b04ae688f.slice/crio-9a2a7a6a1e00d74185f4a8798a5a48d8b4b1dfd7ed9d6a85a4bd3a506c3d545a WatchSource:0}: Error finding container 9a2a7a6a1e00d74185f4a8798a5a48d8b4b1dfd7ed9d6a85a4bd3a506c3d545a: Status 404 returned error can't find the container with id 9a2a7a6a1e00d74185f4a8798a5a48d8b4b1dfd7ed9d6a85a4bd3a506c3d545a
Jul 29 00:03:22.118657 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.118621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:22.205222 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.205131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:22.780762 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.780728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" event={"ID":"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f","Type":"ContainerStarted","Data":"8abdd090f3ccd5acd60b495be0fb488a23d6cbdaad8d834756baaf0e901fa722"}
Jul 29 00:03:22.781279 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.780796    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:22.781279 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.780821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" event={"ID":"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f","Type":"ContainerStarted","Data":"9a2a7a6a1e00d74185f4a8798a5a48d8b4b1dfd7ed9d6a85a4bd3a506c3d545a"}
Jul 29 00:03:22.798824 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:22.798776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" podStartSLOduration=1.316379078 podStartE2EDuration="1.798761806s" totalImagesPullingTime="482.382728ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:03:22.797301883 +0000 UTC m=+2503.324351146" watchObservedRunningTime="2026-07-29 00:03:22.798761806 +0000 UTC m=+2503.325811068"
Jul 29 00:03:28.792900 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:28.792865    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:03:40.430048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:40.430011    2564 scope.go:117] "RemoveContainer" containerID="df719e411aca1317ebef6f27a6fabe99442dce26b3ef5fd61ad65aa4f4a80ee2"
Jul 29 00:03:46.327751 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:46.327717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:03:49.257930 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.257896    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"]
Jul 29 00:03:49.262020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.261994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:49.264629 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.264599    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:03:49.267589 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.267562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"]
Jul 29 00:03:49.317833 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.317788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519e1e-jdt88\" (UID: \"06413e9b-3e6f-4632-8123-5770a9ee1edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:49.418845 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.418805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519e1e-jdt88\" (UID: \"06413e9b-3e6f-4632-8123-5770a9ee1edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:49.419558 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.419535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-519e1e-jdt88\" (UID: \"06413e9b-3e6f-4632-8123-5770a9ee1edd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:49.576137 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.576100    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:49.922289 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.922252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"]
Jul 29 00:03:49.924224 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:03:49.924196    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06413e9b_3e6f_4632_8123_5770a9ee1edd.slice/crio-0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd WatchSource:0}: Error finding container 0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd: Status 404 returned error can't find the container with id 0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd
Jul 29 00:03:49.947065 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:49.947039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88" event={"ID":"06413e9b-3e6f-4632-8123-5770a9ee1edd","Type":"ContainerStarted","Data":"0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd"}
Jul 29 00:03:50.955758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:50.955719    2564 generic.go:356] "Generic (PLEG): container finished" podID="06413e9b-3e6f-4632-8123-5770a9ee1edd" containerID="5a3100de841f59df6319ba8d9cb8c5438b3d898f650b5da79463bc6ea948c394" exitCode=0
Jul 29 00:03:50.956293 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:50.955816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88" event={"ID":"06413e9b-3e6f-4632-8123-5770a9ee1edd","Type":"ContainerDied","Data":"5a3100de841f59df6319ba8d9cb8c5438b3d898f650b5da79463bc6ea948c394"}
Jul 29 00:03:52.111148 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.111120    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:52.246515 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.246414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca\") pod \"06413e9b-3e6f-4632-8123-5770a9ee1edd\" (UID: \"06413e9b-3e6f-4632-8123-5770a9ee1edd\") "
Jul 29 00:03:52.246781 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.246749    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06413e9b-3e6f-4632-8123-5770a9ee1edd" (UID: "06413e9b-3e6f-4632-8123-5770a9ee1edd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:03:52.347610 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.347571    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06413e9b-3e6f-4632-8123-5770a9ee1edd-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:03:52.973688 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.973654    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"
Jul 29 00:03:52.973874 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.973655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88" event={"ID":"06413e9b-3e6f-4632-8123-5770a9ee1edd","Type":"ContainerDied","Data":"0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd"}
Jul 29 00:03:52.973874 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:03:52.973787    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0583e34a087bea6376cf021d616bf89b912db78a2e7156c8c65d7136ef147fcd"
Jul 29 00:04:04.717277 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.717227    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:04.778008 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.777956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:04.877419 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.877373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:04.890070 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.890040    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"]
Jul 29 00:04:04.894831 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.894800    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-519e1e-jdt88"]
Jul 29 00:04:04.938973 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:04.938939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:05.281500 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:05.281466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:06.116330 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:06.116290    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06413e9b-3e6f-4632-8123-5770a9ee1edd" path="/var/lib/kubelet/pods/06413e9b-3e6f-4632-8123-5770a9ee1edd/volumes"
Jul 29 00:04:09.812469 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:09.812430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:09.864714 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:09.864678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:11.090419 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.090377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:11.149301 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.149265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:11.327930 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.327896    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:04:11.328185 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.328154    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" containerID="cri-o://4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890" gracePeriod=30
Jul 29 00:04:11.351092 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.351004    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:11.560228 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.560197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:11.671051 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.670952    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"]
Jul 29 00:04:11.671549 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.671524    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06413e9b-3e6f-4632-8123-5770a9ee1edd" containerName="revoke-keys"
Jul 29 00:04:11.671549 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.671547    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06413e9b-3e6f-4632-8123-5770a9ee1edd" containerName="revoke-keys"
Jul 29 00:04:11.671763 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.671632    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06413e9b-3e6f-4632-8123-5770a9ee1edd" containerName="revoke-keys"
Jul 29 00:04:11.675979 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.675959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:11.678562 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.678539    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:04:11.681323 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.681240    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"]
Jul 29 00:04:11.736575 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.736540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-48f20e-pklzg\" (UID: \"4e7aee48-209b-4d33-ac47-3f926edf8e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:11.838008 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.837970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-48f20e-pklzg\" (UID: \"4e7aee48-209b-4d33-ac47-3f926edf8e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:11.838670 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.838647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-48f20e-pklzg\" (UID: \"4e7aee48-209b-4d33-ac47-3f926edf8e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:11.989583 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:11.989492    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:12.147559 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:12.147524    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"]
Jul 29 00:04:12.149144 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:04:12.149113    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7aee48_209b_4d33_ac47_3f926edf8e04.slice/crio-2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02 WatchSource:0}: Error finding container 2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02: Status 404 returned error can't find the container with id 2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02
Jul 29 00:04:12.830147 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:12.830111    2564 patch_prober.go:28] interesting pod/e2e-api-a-519e1e-openshift-default-548b844756-446z7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 00:04:12.830319 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:12.830167    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:04:13.118876 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:13.118784    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e7aee48-209b-4d33-ac47-3f926edf8e04" containerID="0de4aa23e61e4e7f85c2498c073e297db9959fffaa169fd5bf5c32a04e1e2741" exitCode=0
Jul 29 00:04:13.119020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:13.118877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg" event={"ID":"4e7aee48-209b-4d33-ac47-3f926edf8e04","Type":"ContainerDied","Data":"0de4aa23e61e4e7f85c2498c073e297db9959fffaa169fd5bf5c32a04e1e2741"}
Jul 29 00:04:13.119020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:13.118933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg" event={"ID":"4e7aee48-209b-4d33-ac47-3f926edf8e04","Type":"ContainerStarted","Data":"2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02"}
Jul 29 00:04:14.262777 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:14.262745    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:14.366451 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:14.366334    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca\") pod \"4e7aee48-209b-4d33-ac47-3f926edf8e04\" (UID: \"4e7aee48-209b-4d33-ac47-3f926edf8e04\") "
Jul 29 00:04:14.367564 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:14.367525    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e7aee48-209b-4d33-ac47-3f926edf8e04" (UID: "4e7aee48-209b-4d33-ac47-3f926edf8e04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:04:14.467727 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:14.467634    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e7aee48-209b-4d33-ac47-3f926edf8e04-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:15.131864 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:15.131836    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"
Jul 29 00:04:15.132020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:15.131861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg" event={"ID":"4e7aee48-209b-4d33-ac47-3f926edf8e04","Type":"ContainerDied","Data":"2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02"}
Jul 29 00:04:15.132020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:15.131904    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b593ee90e278f70288d6231e568da0fd5cc690421ebace296b61f73c9a5bb02"
Jul 29 00:04:16.605410 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.605382    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:04:16.686532 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686532 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686538    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686754 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686581    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686754 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686673    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686863 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686752    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fbzz\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686863 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686804    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.686863 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686851    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.687023 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686874    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:16.687023 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686899    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.687023 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.686951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert\") pod \"0853acaf-0076-4b85-913f-1a5bcf257f23\" (UID: \"0853acaf-0076-4b85-913f-1a5bcf257f23\") "
Jul 29 00:04:16.687202 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687061    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:16.687369 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687328    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:16.687488 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687401    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.687488 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687418    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data" (OuterVolumeSpecName: "istio-data") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:16.687488 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687426    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.687710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.687522    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:04:16.688943 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.688918    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:04:16.689109 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.689088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:16.689266 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.689243    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token" (OuterVolumeSpecName: "istio-token") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:16.689325 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.689250    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz" (OuterVolumeSpecName: "kube-api-access-6fbzz") pod "0853acaf-0076-4b85-913f-1a5bcf257f23" (UID: "0853acaf-0076-4b85-913f-1a5bcf257f23"). InnerVolumeSpecName "kube-api-access-6fbzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:16.788525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788485    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788518    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788528    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0853acaf-0076-4b85-913f-1a5bcf257f23-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788537    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788794 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788545    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788794 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788552    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0853acaf-0076-4b85-913f-1a5bcf257f23-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:16.788794 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:16.788560    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fbzz\" (UniqueName: \"kubernetes.io/projected/0853acaf-0076-4b85-913f-1a5bcf257f23-kube-api-access-6fbzz\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:17.147071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.147037    2564 generic.go:356] "Generic (PLEG): container finished" podID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerID="4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890" exitCode=0
Jul 29 00:04:17.147243 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.147093    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"
Jul 29 00:04:17.147243 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.147113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerDied","Data":"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"}
Jul 29 00:04:17.147243 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.147172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7" event={"ID":"0853acaf-0076-4b85-913f-1a5bcf257f23","Type":"ContainerDied","Data":"67bb98d266b0f47f2aaf50cf5279aec2d15e9ba42641e795f7dc08d22b842a97"}
Jul 29 00:04:17.147243 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.147195    2564 scope.go:117] "RemoveContainer" containerID="4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"
Jul 29 00:04:17.157711 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.157692    2564 scope.go:117] "RemoveContainer" containerID="e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"
Jul 29 00:04:17.166531 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.166504    2564 scope.go:117] "RemoveContainer" containerID="4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"
Jul 29 00:04:17.166923 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:04:17.166886    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890\": container with ID starting with 4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890 not found: ID does not exist" containerID="4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"
Jul 29 00:04:17.166923 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.166926    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890"} err="failed to get container status \"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890\": rpc error: code = NotFound desc = could not find container \"4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890\": container with ID starting with 4b84b85b39d1313cb05d8024b06044ef5f719c0855256bfc62f0ec51d4d86890 not found: ID does not exist"
Jul 29 00:04:17.167124 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.166953    2564 scope.go:117] "RemoveContainer" containerID="e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"
Jul 29 00:04:17.167455 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:04:17.167323    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b\": container with ID starting with e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b not found: ID does not exist" containerID="e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"
Jul 29 00:04:17.167729 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.167700    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b"} err="failed to get container status \"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b\": rpc error: code = NotFound desc = could not find container \"e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b\": container with ID starting with e401734f6f48ed0f9337a1006c2724984311ba7077ea4d5de4c9847bf11e0a1b not found: ID does not exist"
Jul 29 00:04:17.170735 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.170716    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:04:17.175241 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:17.175212    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-519e1e-openshift-default-548b844756-446z7"]
Jul 29 00:04:18.120913 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:18.120879    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" path="/var/lib/kubelet/pods/0853acaf-0076-4b85-913f-1a5bcf257f23/volumes"
Jul 29 00:04:27.092588 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.092550    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:04:27.093194 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.092868    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" podUID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" containerName="maas-api" containerID="cri-o://8abdd090f3ccd5acd60b495be0fb488a23d6cbdaad8d834756baaf0e901fa722" gracePeriod=30
Jul 29 00:04:27.116885 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.116845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:27.181269 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.181209    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:27.212336 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.212304    2564 generic.go:356] "Generic (PLEG): container finished" podID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" containerID="8abdd090f3ccd5acd60b495be0fb488a23d6cbdaad8d834756baaf0e901fa722" exitCode=0
Jul 29 00:04:27.212476 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.212412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" event={"ID":"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f","Type":"ContainerDied","Data":"8abdd090f3ccd5acd60b495be0fb488a23d6cbdaad8d834756baaf0e901fa722"}
Jul 29 00:04:27.241875 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.241829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:27.281775 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.281699    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"]
Jul 29 00:04:27.284997 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.284969    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-48f20e-pklzg"]
Jul 29 00:04:27.302059 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.302018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:27.353887 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.353818    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:04:27.492961 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.492929    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca\") pod \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") "
Jul 29 00:04:27.493138 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.492997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls\") pod \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") "
Jul 29 00:04:27.493138 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.493090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlt6s\" (UniqueName: \"kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s\") pod \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\" (UID: \"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f\") "
Jul 29 00:04:27.493356 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.493321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" (UID: "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:04:27.495086 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.495056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" (UID: "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:04:27.495201 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.495122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s" (OuterVolumeSpecName: "kube-api-access-rlt6s") pod "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" (UID: "7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f"). InnerVolumeSpecName "kube-api-access-rlt6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:27.594528 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.594485    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlt6s\" (UniqueName: \"kubernetes.io/projected/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-kube-api-access-rlt6s\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:27.594528 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.594520    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:27.594528 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.594531    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:04:27.672267 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:27.672168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:28.116050 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.116010    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7aee48-209b-4d33-ac47-3f926edf8e04" path="/var/lib/kubelet/pods/4e7aee48-209b-4d33-ac47-3f926edf8e04/volumes"
Jul 29 00:04:28.221091 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.221057    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"
Jul 29 00:04:28.221269 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.221099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb" event={"ID":"7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f","Type":"ContainerDied","Data":"9a2a7a6a1e00d74185f4a8798a5a48d8b4b1dfd7ed9d6a85a4bd3a506c3d545a"}
Jul 29 00:04:28.221269 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.221164    2564 scope.go:117] "RemoveContainer" containerID="8abdd090f3ccd5acd60b495be0fb488a23d6cbdaad8d834756baaf0e901fa722"
Jul 29 00:04:28.237707 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.237680    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:04:28.240894 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:28.240870    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-48f20e-556fb44b9f-b2ldb"]
Jul 29 00:04:30.116365 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:30.116311    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" path="/var/lib/kubelet/pods/7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f/volumes"
Jul 29 00:04:33.448147 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:33.448107    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:33.549110 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:33.549068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:33.731538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:33.731413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:33.943361 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:33.943311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:34.274874 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.274839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:34.555266 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555234    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:04:34.555796 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555779    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e7aee48-209b-4d33-ac47-3f926edf8e04" containerName="revoke-keys"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555799    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7aee48-209b-4d33-ac47-3f926edf8e04" containerName="revoke-keys"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555815    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555820    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555831    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555837    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.555848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555845    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" containerName="maas-api"
Jul 29 00:04:34.556048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555850    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" containerName="maas-api"
Jul 29 00:04:34.556048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555913    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.556048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555920    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0853acaf-0076-4b85-913f-1a5bcf257f23" containerName="istio-proxy"
Jul 29 00:04:34.556048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555928    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a89eb3e-9a2d-487e-a1e4-7f7b04ae688f" containerName="maas-api"
Jul 29 00:04:34.556048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.555935    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e7aee48-209b-4d33-ac47-3f926edf8e04" containerName="revoke-keys"
Jul 29 00:04:34.560699 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.560664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.563228 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.563202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-daedc5-openshift-default-dockercfg-92ccc\""
Jul 29 00:04:34.570172 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.570144    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:04:34.579991 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.579959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:34.659158 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659513 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj4qj\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659661 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659661 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659915 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.659915 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.659893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.682383 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.682316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:34.760721 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760721 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj4qj\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.760972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.760917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.761434 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.761403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.761561 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.761462    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.761561 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.761526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.761561 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.761552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.761746 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.761725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.763394 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.763374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.763826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.763804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.773674 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.773647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj4qj\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.776397 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.776375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token\") pod \"e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") " pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:34.873864 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:34.873773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:35.022045 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.022017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:04:35.024183 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:04:35.024152    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb251d90e_3ac2_4e8e_9524_1998bfb255c7.slice/crio-303524c22174e8f5c8099b7a8c378b477cd444be2a3ad84d9394cadb4e678839 WatchSource:0}: Error finding container 303524c22174e8f5c8099b7a8c378b477cd444be2a3ad84d9394cadb4e678839: Status 404 returned error can't find the container with id 303524c22174e8f5c8099b7a8c378b477cd444be2a3ad84d9394cadb4e678839
Jul 29 00:04:35.026259 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.026224    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:35.026364 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.026295    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:35.026364 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.026325    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:35.270067 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.269948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerStarted","Data":"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"}
Jul 29 00:04:35.270067 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.270013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerStarted","Data":"303524c22174e8f5c8099b7a8c378b477cd444be2a3ad84d9394cadb4e678839"}
Jul 29 00:04:35.291904 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.291838    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podStartSLOduration=1.291812759 podStartE2EDuration="1.291812759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:04:35.288242211 +0000 UTC m=+2575.815291475" watchObservedRunningTime="2026-07-29 00:04:35.291812759 +0000 UTC m=+2575.818862023"
Jul 29 00:04:35.874398 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:35.874334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:36.879219 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:36.879185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:37.183506 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.183382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:37.283718 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.283685    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:37.284940 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.284916    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:37.288017 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.287989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:37.366135 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.366099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:37.758996 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.758955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:37.867060 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.867012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:37.949091 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:37.949053    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:04:51.377764 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.377730    2564 generic.go:356] "Generic (PLEG): container finished" podID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerID="102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83" exitCode=0
Jul 29 00:04:51.378177 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.377805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerDied","Data":"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"}
Jul 29 00:04:51.378246 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.378231    2564 scope.go:117] "RemoveContainer" containerID="102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"
Jul 29 00:04:51.379493 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.379459    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:51.379588 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.379550    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:51.379626 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:51.379591    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:04:52.283826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:52.283783    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:52.390011 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:52.389974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerStarted","Data":"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"}
Jul 29 00:04:52.875234 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:52.875200    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:52.876743 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:52.876710    2564 patch_prober.go:28] interesting pod/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 29 00:04:52.877026 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:52.876998    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 29 00:04:53.874849 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:53.874797    2564 patch_prober.go:28] interesting pod/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 29 00:04:53.875406 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:53.874874    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 29 00:04:54.874835 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:54.874790    2564 patch_prober.go:28] interesting pod/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 29 00:04:54.875059 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:54.874865    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 29 00:04:55.874647 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:55.874598    2564 patch_prober.go:28] interesting pod/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Jul 29 00:04:55.874867 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:55.874671    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Jul 29 00:04:56.878761 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:56.878731    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:56.879259 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:56.879109    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:04:56.879743 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:04:56.879724    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:05:02.298735 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:02.298637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:02.539618 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:02.539577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:02.854073 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:02.854038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:03.287999 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:03.287916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:05.806053 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:05.806017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:05.888918 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:05.888875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:05.965841 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:05.965776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:06.065319 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:06.065222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:06.489398 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:06.489283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:06.588692 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:06.588657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:30.765604 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:30.765563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:47.668176 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:47.668141    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:49.206476 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:49.206440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:05:56.415732 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:05:56.415667    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:06:36.758383 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:36.758322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:06:40.305594 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:40.305561    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:06:40.310519 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:40.310492    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:06:40.312210 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:40.312186    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:06:40.316741 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:40.316722    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:06:40.488848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:06:40.488809    2564 scope.go:117] "RemoveContainer" containerID="139262acb264f72d0739d6af7756b1e236a90be8f5a75d6b7be12c4934f53dcd"
Jul 29 00:07:32.078033 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:07:32.077990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:08:40.521694 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:08:40.521660    2564 scope.go:117] "RemoveContainer" containerID="54eb5ec55322190ff9423b528d0b1b7aedc35373b3f7ab7c42abf5e515c47b1e"
Jul 29 00:08:40.531207 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:08:40.531179    2564 scope.go:117] "RemoveContainer" containerID="5e5437e8df515b652e9e2dcd6d07c30418b832ee36934a2986370d4e6f416256"
Jul 29 00:09:13.740222 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:09:13.740182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:09:15.421465 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:09:15.421414    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:10:40.570974 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:10:40.570928    2564 scope.go:117] "RemoveContainer" containerID="0de4aa23e61e4e7f85c2498c073e297db9959fffaa169fd5bf5c32a04e1e2741"
Jul 29 00:10:40.581547 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:10:40.581517    2564 scope.go:117] "RemoveContainer" containerID="5a3100de841f59df6319ba8d9cb8c5438b3d898f650b5da79463bc6ea948c394"
Jul 29 00:11:00.427179 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:00.427145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:11:01.361774 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:01.361728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:11:17.692792 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:17.692754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:11:18.547882 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:18.547845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:11:40.345540 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:40.345513    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:11:40.355397 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:40.355372    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:11:40.356092 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:40.356069    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:11:40.362916 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:40.362896    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:11:52.599470 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:11:52.599426    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:12:47.158760 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:12:47.158678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:12:56.741806 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:12:56.741770    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:12:57.643646 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:12:57.643613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:13:10.045846 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:10.045811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:13:35.143529 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:35.143470    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:13:53.480528 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.480487    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:13:53.484318 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.484293    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:13:53.494644 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.494615    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:13:53.640116 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.640072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlfk5\" (UniqueName: \"kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5\") pod \"kuadrant-operator-catalog-wwzh6\" (UID: \"21f22f97-10bf-490a-9add-5d57a7a7cd46\") " pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:13:53.741648 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.741556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlfk5\" (UniqueName: \"kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5\") pod \"kuadrant-operator-catalog-wwzh6\" (UID: \"21f22f97-10bf-490a-9add-5d57a7a7cd46\") " pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:13:53.750655 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.750618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlfk5\" (UniqueName: \"kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5\") pod \"kuadrant-operator-catalog-wwzh6\" (UID: \"21f22f97-10bf-490a-9add-5d57a7a7cd46\") " pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:13:53.795813 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.795776    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:13:53.939230 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.939197    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:13:53.941030 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:13:53.941002    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f22f97_10bf_490a_9add_5d57a7a7cd46.slice/crio-756c189db6468444dfc7366cb7dbc50082410763497eec39a0531b1f5ddee2da WatchSource:0}: Error finding container 756c189db6468444dfc7366cb7dbc50082410763497eec39a0531b1f5ddee2da: Status 404 returned error can't find the container with id 756c189db6468444dfc7366cb7dbc50082410763497eec39a0531b1f5ddee2da
Jul 29 00:13:53.942870 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:53.942845    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:13:54.704093 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:54.703998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" event={"ID":"21f22f97-10bf-490a-9add-5d57a7a7cd46","Type":"ContainerStarted","Data":"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"}
Jul 29 00:13:54.704093 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:54.704056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" event={"ID":"21f22f97-10bf-490a-9add-5d57a7a7cd46","Type":"ContainerStarted","Data":"756c189db6468444dfc7366cb7dbc50082410763497eec39a0531b1f5ddee2da"}
Jul 29 00:13:54.722095 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:13:54.722040    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" podStartSLOduration=1.249879695 podStartE2EDuration="1.722023311s" totalImagesPullingTime="472.143616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:13:54.718871458 +0000 UTC m=+3135.245920723" watchObservedRunningTime="2026-07-29 00:13:54.722023311 +0000 UTC m=+3135.249072573"
Jul 29 00:14:03.796557 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:03.796467    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:03.796557 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:03.796547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:03.819441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:03.819414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:04.794228 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:04.794195    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:04.827538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:04.827497    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:14:06.779698 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:06.779632    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" podUID="21f22f97-10bf-490a-9add-5d57a7a7cd46" containerName="registry-server" containerID="cri-o://9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19" gracePeriod=2
Jul 29 00:14:07.049984 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.049949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:07.073851 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.073817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlfk5\" (UniqueName: \"kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5\") pod \"21f22f97-10bf-490a-9add-5d57a7a7cd46\" (UID: \"21f22f97-10bf-490a-9add-5d57a7a7cd46\") "
Jul 29 00:14:07.075937 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.075907    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5" (OuterVolumeSpecName: "kube-api-access-wlfk5") pod "21f22f97-10bf-490a-9add-5d57a7a7cd46" (UID: "21f22f97-10bf-490a-9add-5d57a7a7cd46"). InnerVolumeSpecName "kube-api-access-wlfk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:14:07.174629 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.174583    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlfk5\" (UniqueName: \"kubernetes.io/projected/21f22f97-10bf-490a-9add-5d57a7a7cd46-kube-api-access-wlfk5\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:14:07.786745 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.786715    2564 generic.go:356] "Generic (PLEG): container finished" podID="21f22f97-10bf-490a-9add-5d57a7a7cd46" containerID="9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19" exitCode=0
Jul 29 00:14:07.787183 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.786768    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6"
Jul 29 00:14:07.787183 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.786802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" event={"ID":"21f22f97-10bf-490a-9add-5d57a7a7cd46","Type":"ContainerDied","Data":"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"}
Jul 29 00:14:07.787183 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.786851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wwzh6" event={"ID":"21f22f97-10bf-490a-9add-5d57a7a7cd46","Type":"ContainerDied","Data":"756c189db6468444dfc7366cb7dbc50082410763497eec39a0531b1f5ddee2da"}
Jul 29 00:14:07.787183 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.786873    2564 scope.go:117] "RemoveContainer" containerID="9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"
Jul 29 00:14:07.796537 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.796516    2564 scope.go:117] "RemoveContainer" containerID="9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"
Jul 29 00:14:07.796899 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:14:07.796879    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19\": container with ID starting with 9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19 not found: ID does not exist" containerID="9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"
Jul 29 00:14:07.796965 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.796913    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19"} err="failed to get container status \"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19\": rpc error: code = NotFound desc = could not find container \"9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19\": container with ID starting with 9fff8cded5057d2a3ae650af62e2d7ad5e666257ca609fe63c21853b7f7b0b19 not found: ID does not exist"
Jul 29 00:14:07.809038 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.809010    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:14:07.811556 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:07.811535    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wwzh6"]
Jul 29 00:14:08.115842 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:08.115807    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f22f97-10bf-490a-9add-5d57a7a7cd46" path="/var/lib/kubelet/pods/21f22f97-10bf-490a-9add-5d57a7a7cd46/volumes"
Jul 29 00:14:51.336455 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:51.336421    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:14:52.780465 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:52.780430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:14:53.292958 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.292909    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210833584"]
Jul 29 00:14:53.293411 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.293396    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f22f97-10bf-490a-9add-5d57a7a7cd46" containerName="registry-server"
Jul 29 00:14:53.293411 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.293411    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f22f97-10bf-490a-9add-5d57a7a7cd46" containerName="registry-server"
Jul 29 00:14:53.293520 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.293483    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f22f97-10bf-490a-9add-5d57a7a7cd46" containerName="registry-server"
Jul 29 00:14:53.296571 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.296552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:53.303581 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.303562    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9rj9\""
Jul 29 00:14:53.314509 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.314481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210833584"]
Jul 29 00:14:53.418289 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.418240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4trr\" (UniqueName: \"kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr\") pod \"test-curl-7401-140364210833584\" (UID: \"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d\") " pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:53.519004 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.518964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4trr\" (UniqueName: \"kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr\") pod \"test-curl-7401-140364210833584\" (UID: \"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d\") " pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:53.531534 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.531500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4trr\" (UniqueName: \"kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr\") pod \"test-curl-7401-140364210833584\" (UID: \"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d\") " pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:53.605801 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.605691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:53.754480 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:53.754449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210833584"]
Jul 29 00:14:53.756894 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:14:53.756866    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0d92e22_36ff_4e9c_8e93_7020a4d4ae6d.slice/crio-e1dcbe3ceee7223e0448fa0576546f492e652332802c35d2027949639e281e04 WatchSource:0}: Error finding container e1dcbe3ceee7223e0448fa0576546f492e652332802c35d2027949639e281e04: Status 404 returned error can't find the container with id e1dcbe3ceee7223e0448fa0576546f492e652332802c35d2027949639e281e04
Jul 29 00:14:54.088269 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:54.088226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210833584" event={"ID":"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d","Type":"ContainerStarted","Data":"e1dcbe3ceee7223e0448fa0576546f492e652332802c35d2027949639e281e04"}
Jul 29 00:14:55.096040 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:55.096000    2564 generic.go:356] "Generic (PLEG): container finished" podID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" containerID="cd93bc001a14232e92172af8aa182e274ef42eebb1519436d1e39f9f6f4da41f" exitCode=0
Jul 29 00:14:55.096494 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:55.096091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210833584" event={"ID":"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d","Type":"ContainerDied","Data":"cd93bc001a14232e92172af8aa182e274ef42eebb1519436d1e39f9f6f4da41f"}
Jul 29 00:14:56.246679 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.246649    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:56.268421 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.268390    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210833584_d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d/test-curl-7401-140364210833584/0.log"
Jul 29 00:14:56.300873 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.300841    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210833584"]
Jul 29 00:14:56.303244 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.303219    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210833584"]
Jul 29 00:14:56.349500 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.349464    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4trr\" (UniqueName: \"kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr\") pod \"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d\" (UID: \"d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d\") "
Jul 29 00:14:56.351681 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.351656    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr" (OuterVolumeSpecName: "kube-api-access-d4trr") pod "d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" (UID: "d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d"). InnerVolumeSpecName "kube-api-access-d4trr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:14:56.438200 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.438166    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210762416"]
Jul 29 00:14:56.438658 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.438643    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" containerName="test-curl-7401-140364210833584"
Jul 29 00:14:56.438710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.438660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" containerName="test-curl-7401-140364210833584"
Jul 29 00:14:56.438744 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.438728    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" containerName="test-curl-7401-140364210833584"
Jul 29 00:14:56.441933 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.441916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:56.447403 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.447377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210762416"]
Jul 29 00:14:56.450497 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.450468    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4trr\" (UniqueName: \"kubernetes.io/projected/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d-kube-api-access-d4trr\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:14:56.551389 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.551327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lc99\" (UniqueName: \"kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99\") pod \"test-curl-7401-140364210762416\" (UID: \"2c9de9d1-05c3-4648-94b1-54cbe5a33255\") " pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:56.652618 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.652523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lc99\" (UniqueName: \"kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99\") pod \"test-curl-7401-140364210762416\" (UID: \"2c9de9d1-05c3-4648-94b1-54cbe5a33255\") " pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:56.661996 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.661966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lc99\" (UniqueName: \"kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99\") pod \"test-curl-7401-140364210762416\" (UID: \"2c9de9d1-05c3-4648-94b1-54cbe5a33255\") " pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:56.752120 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.752081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:56.899315 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:56.899281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210762416"]
Jul 29 00:14:56.900564 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:14:56.900537    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c9de9d1_05c3_4648_94b1_54cbe5a33255.slice/crio-098ca28f7c4f5065437f426b4acb52e47e9522e6762dc557e8e5790601ede006 WatchSource:0}: Error finding container 098ca28f7c4f5065437f426b4acb52e47e9522e6762dc557e8e5790601ede006: Status 404 returned error can't find the container with id 098ca28f7c4f5065437f426b4acb52e47e9522e6762dc557e8e5790601ede006
Jul 29 00:14:57.110151 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:57.110109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210762416" event={"ID":"2c9de9d1-05c3-4648-94b1-54cbe5a33255","Type":"ContainerStarted","Data":"098ca28f7c4f5065437f426b4acb52e47e9522e6762dc557e8e5790601ede006"}
Jul 29 00:14:57.111539 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:57.111516    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:57.111539 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:57.111529    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1dcbe3ceee7223e0448fa0576546f492e652332802c35d2027949639e281e04"
Jul 29 00:14:57.122472 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:14:57.122443    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7401-140364210833584\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" pod="opendatahub/test-curl-7401-140364210833584"
Jul 29 00:14:58.116737 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:58.116704    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d" path="/var/lib/kubelet/pods/d0d92e22-36ff-4e9c-8e93-7020a4d4ae6d/volumes"
Jul 29 00:14:58.118479 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:58.118452    2564 generic.go:356] "Generic (PLEG): container finished" podID="2c9de9d1-05c3-4648-94b1-54cbe5a33255" containerID="16d92f911eaf007d15f0a3fa5d0301119829317bfabf4c53d517223a58c8af63" exitCode=0
Jul 29 00:14:58.118593 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:58.118515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210762416" event={"ID":"2c9de9d1-05c3-4648-94b1-54cbe5a33255","Type":"ContainerDied","Data":"16d92f911eaf007d15f0a3fa5d0301119829317bfabf4c53d517223a58c8af63"}
Jul 29 00:14:59.272241 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.272207    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:14:59.291405 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.291373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210762416_2c9de9d1-05c3-4648-94b1-54cbe5a33255/test-curl-7401-140364210762416/0.log"
Jul 29 00:14:59.317157 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.317118    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210762416"]
Jul 29 00:14:59.323457 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.323427    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210762416"]
Jul 29 00:14:59.378539 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.378435    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lc99\" (UniqueName: \"kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99\") pod \"2c9de9d1-05c3-4648-94b1-54cbe5a33255\" (UID: \"2c9de9d1-05c3-4648-94b1-54cbe5a33255\") "
Jul 29 00:14:59.380788 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.380755    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99" (OuterVolumeSpecName: "kube-api-access-2lc99") pod "2c9de9d1-05c3-4648-94b1-54cbe5a33255" (UID: "2c9de9d1-05c3-4648-94b1-54cbe5a33255"). InnerVolumeSpecName "kube-api-access-2lc99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:14:59.479981 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.479941    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lc99\" (UniqueName: \"kubernetes.io/projected/2c9de9d1-05c3-4648-94b1-54cbe5a33255-kube-api-access-2lc99\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:14:59.516569 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.516532    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:14:59.517000 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.516986    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9de9d1-05c3-4648-94b1-54cbe5a33255" containerName="test-curl-7401-140364210762416"
Jul 29 00:14:59.517055 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.517002    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9de9d1-05c3-4648-94b1-54cbe5a33255" containerName="test-curl-7401-140364210762416"
Jul 29 00:14:59.517092 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.517079    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9de9d1-05c3-4648-94b1-54cbe5a33255" containerName="test-curl-7401-140364210762416"
Jul 29 00:14:59.520166 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.520149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:14:59.528301 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.528039    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:14:59.581442 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.581396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b886m\" (UniqueName: \"kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m\") pod \"test-curl-7401-140364210698256\" (UID: \"3dbe86e4-c5f4-41c5-861a-fc624da2f122\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:14:59.682817 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.682701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b886m\" (UniqueName: \"kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m\") pod \"test-curl-7401-140364210698256\" (UID: \"3dbe86e4-c5f4-41c5-861a-fc624da2f122\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:14:59.694946 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.694912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b886m\" (UniqueName: \"kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m\") pod \"test-curl-7401-140364210698256\" (UID: \"3dbe86e4-c5f4-41c5-861a-fc624da2f122\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:14:59.831197 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.831151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:14:59.987538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:14:59.987502    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:14:59.990335 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:14:59.990297    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dbe86e4_c5f4_41c5_861a_fc624da2f122.slice/crio-a8066d05bd422a1abdf5e64114a17d023bc5637819efda3baa5226b9b72e14fe WatchSource:0}: Error finding container a8066d05bd422a1abdf5e64114a17d023bc5637819efda3baa5226b9b72e14fe: Status 404 returned error can't find the container with id a8066d05bd422a1abdf5e64114a17d023bc5637819efda3baa5226b9b72e14fe
Jul 29 00:15:00.116357 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.116311    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c9de9d1-05c3-4648-94b1-54cbe5a33255" path="/var/lib/kubelet/pods/2c9de9d1-05c3-4648-94b1-54cbe5a33255/volumes"
Jul 29 00:15:00.130782 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.130756    2564 scope.go:117] "RemoveContainer" containerID="16d92f911eaf007d15f0a3fa5d0301119829317bfabf4c53d517223a58c8af63"
Jul 29 00:15:00.130902 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.130760    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210762416"
Jul 29 00:15:00.132150 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.132063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210698256" event={"ID":"3dbe86e4-c5f4-41c5-861a-fc624da2f122","Type":"ContainerStarted","Data":"a8066d05bd422a1abdf5e64114a17d023bc5637819efda3baa5226b9b72e14fe"}
Jul 29 00:15:00.144889 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.144865    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"]
Jul 29 00:15:00.149991 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.149971    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"
Jul 29 00:15:00.152826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.152803    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"]
Jul 29 00:15:00.156212 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.156152    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"]
Jul 29 00:15:00.156468 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.156444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"
Jul 29 00:15:00.158054 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.157938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:15:00.160223 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.160205    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"
Jul 29 00:15:00.160414 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.160388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"]
Jul 29 00:15:00.164941 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.164922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"
Jul 29 00:15:00.172442 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.172408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"]
Jul 29 00:15:00.177030 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.176642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"
Jul 29 00:15:00.178649 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.178455    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"]
Jul 29 00:15:00.189879 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.189853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"
Jul 29 00:15:00.354156 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:00.353459    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac817f8_bcac_44c6_abe8_2ca130146abb.slice/crio-6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2 WatchSource:0}: Error finding container 6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2: Status 404 returned error can't find the container with id 6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2
Jul 29 00:15:00.354156 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.354049    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"]
Jul 29 00:15:00.594069 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.594032    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"]
Jul 29 00:15:00.594290 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:00.594267    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a27f93_4155_495a_911e_1467eacc303d.slice/crio-ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f WatchSource:0}: Error finding container ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f: Status 404 returned error can't find the container with id ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f
Jul 29 00:15:00.620526 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:00.620500    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"]
Jul 29 00:15:00.626656 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:00.626629    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936068a2_27a7_49aa_8a22_1410d8d069b4.slice/crio-1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35 WatchSource:0}: Error finding container 1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35: Status 404 returned error can't find the container with id 1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35
Jul 29 00:15:01.141849 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.141750    2564 generic.go:356] "Generic (PLEG): container finished" podID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" containerID="23036def856e6dc3e59af42ee0fafeadb7ca040b6a32ae0ccacb4c50c2ca3eca" exitCode=0
Jul 29 00:15:01.141849 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.141800    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210698256" event={"ID":"3dbe86e4-c5f4-41c5-861a-fc624da2f122","Type":"ContainerDied","Data":"23036def856e6dc3e59af42ee0fafeadb7ca040b6a32ae0ccacb4c50c2ca3eca"}
Jul 29 00:15:01.143381 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.143333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj" event={"ID":"936068a2-27a7-49aa-8a22-1410d8d069b4","Type":"ContainerStarted","Data":"5d8b14b655c0e3bd4267f6f24a257012706316dfb4d57c0c05ae1d235e3aa0e6"}
Jul 29 00:15:01.143519 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.143402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj" event={"ID":"936068a2-27a7-49aa-8a22-1410d8d069b4","Type":"ContainerStarted","Data":"1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35"}
Jul 29 00:15:01.144902 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.144881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b" event={"ID":"fac817f8-bcac-44c6-abe8-2ca130146abb","Type":"ContainerStarted","Data":"020b441241318f3dbbc290200f633c27efe899ffcc1ced1d4f2274883521c61d"}
Jul 29 00:15:01.145035 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.144919    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b" event={"ID":"fac817f8-bcac-44c6-abe8-2ca130146abb","Type":"ContainerStarted","Data":"6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2"}
Jul 29 00:15:01.146358 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.146323    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc" event={"ID":"36a27f93-4155-495a-911e-1467eacc303d","Type":"ContainerStarted","Data":"9194dcddd044d3ddf893fd048d1036c3cd59d2fdcc90498ea9bc6d0b6d5e74b5"}
Jul 29 00:15:01.146473 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.146380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc" event={"ID":"36a27f93-4155-495a-911e-1467eacc303d","Type":"ContainerStarted","Data":"ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f"}
Jul 29 00:15:01.180418 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.180335    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc" podStartSLOduration=1.1803181409999999 podStartE2EDuration="1.180318141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:01.177991396 +0000 UTC m=+3201.705040661" watchObservedRunningTime="2026-07-29 00:15:01.180318141 +0000 UTC m=+3201.707367404"
Jul 29 00:15:01.194838 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.194789    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b" podStartSLOduration=1.194773698 podStartE2EDuration="1.194773698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:01.192183739 +0000 UTC m=+3201.719233016" watchObservedRunningTime="2026-07-29 00:15:01.194773698 +0000 UTC m=+3201.721822961"
Jul 29 00:15:01.210750 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:01.210695    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj" podStartSLOduration=1.21067722 podStartE2EDuration="1.21067722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:01.207521316 +0000 UTC m=+3201.734570580" watchObservedRunningTime="2026-07-29 00:15:01.21067722 +0000 UTC m=+3201.737726484"
Jul 29 00:15:02.155328 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.155285    2564 generic.go:356] "Generic (PLEG): container finished" podID="936068a2-27a7-49aa-8a22-1410d8d069b4" containerID="5d8b14b655c0e3bd4267f6f24a257012706316dfb4d57c0c05ae1d235e3aa0e6" exitCode=0
Jul 29 00:15:02.155785 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.155386    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj" event={"ID":"936068a2-27a7-49aa-8a22-1410d8d069b4","Type":"ContainerDied","Data":"5d8b14b655c0e3bd4267f6f24a257012706316dfb4d57c0c05ae1d235e3aa0e6"}
Jul 29 00:15:02.156917 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.156889    2564 generic.go:356] "Generic (PLEG): container finished" podID="fac817f8-bcac-44c6-abe8-2ca130146abb" containerID="020b441241318f3dbbc290200f633c27efe899ffcc1ced1d4f2274883521c61d" exitCode=0
Jul 29 00:15:02.157030 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.156966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b" event={"ID":"fac817f8-bcac-44c6-abe8-2ca130146abb","Type":"ContainerDied","Data":"020b441241318f3dbbc290200f633c27efe899ffcc1ced1d4f2274883521c61d"}
Jul 29 00:15:02.158599 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.158570    2564 generic.go:356] "Generic (PLEG): container finished" podID="36a27f93-4155-495a-911e-1467eacc303d" containerID="9194dcddd044d3ddf893fd048d1036c3cd59d2fdcc90498ea9bc6d0b6d5e74b5" exitCode=0
Jul 29 00:15:02.158599 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.158598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc" event={"ID":"36a27f93-4155-495a-911e-1467eacc303d","Type":"ContainerDied","Data":"9194dcddd044d3ddf893fd048d1036c3cd59d2fdcc90498ea9bc6d0b6d5e74b5"}
Jul 29 00:15:02.303470 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.303433    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.337987 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.337954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210698256_3dbe86e4-c5f4-41c5-861a-fc624da2f122/test-curl-7401-140364210698256/0.log"
Jul 29 00:15:02.366878 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.366839    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:02.369756 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.369723    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:02.410315 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.410230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b886m\" (UniqueName: \"kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m\") pod \"3dbe86e4-c5f4-41c5-861a-fc624da2f122\" (UID: \"3dbe86e4-c5f4-41c5-861a-fc624da2f122\") "
Jul 29 00:15:02.412379 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.412355    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m" (OuterVolumeSpecName: "kube-api-access-b886m") pod "3dbe86e4-c5f4-41c5-861a-fc624da2f122" (UID: "3dbe86e4-c5f4-41c5-861a-fc624da2f122"). InnerVolumeSpecName "kube-api-access-b886m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:02.497022 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.496986    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:02.497476 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.497462    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:02.497536 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.497478    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:02.497575 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.497555    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:02.500719 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.500700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.505251 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.505223    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:02.511386 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.511361    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b886m\" (UniqueName: \"kubernetes.io/projected/3dbe86e4-c5f4-41c5-861a-fc624da2f122-kube-api-access-b886m\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:15:02.612222 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.612188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwhjj\" (UniqueName: \"kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj\") pod \"test-curl-7401-140364210698256\" (UID: \"35e05bea-de04-43cd-bd5a-c1916e4d0fe0\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.713078 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.712979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhjj\" (UniqueName: \"kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj\") pod \"test-curl-7401-140364210698256\" (UID: \"35e05bea-de04-43cd-bd5a-c1916e4d0fe0\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.722501 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.722468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhjj\" (UniqueName: \"kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj\") pod \"test-curl-7401-140364210698256\" (UID: \"35e05bea-de04-43cd-bd5a-c1916e4d0fe0\") " pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.811761 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.811719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:02.950647 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:02.950621    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:02.952770 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:02.952738    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35e05bea_de04_43cd_bd5a_c1916e4d0fe0.slice/crio-14553b21045d8e8e26dcb95fda923d7fa6e5d0f4db3934b73af37c2a4ed5e60f WatchSource:0}: Error finding container 14553b21045d8e8e26dcb95fda923d7fa6e5d0f4db3934b73af37c2a4ed5e60f: Status 404 returned error can't find the container with id 14553b21045d8e8e26dcb95fda923d7fa6e5d0f4db3934b73af37c2a4ed5e60f
Jul 29 00:15:03.167509 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.167456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210698256" event={"ID":"35e05bea-de04-43cd-bd5a-c1916e4d0fe0","Type":"ContainerStarted","Data":"14553b21045d8e8e26dcb95fda923d7fa6e5d0f4db3934b73af37c2a4ed5e60f"}
Jul 29 00:15:03.169000 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.168969    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:03.169136 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.169014    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8066d05bd422a1abdf5e64114a17d023bc5637819efda3baa5226b9b72e14fe"
Jul 29 00:15:03.184065 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.184013    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7401-140364210698256" oldPodUID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" podUID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0"
Jul 29 00:15:03.334855 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.334826    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"
Jul 29 00:15:03.412582 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.412553    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"
Jul 29 00:15:03.415522 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:03.415500    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"
Jul 29 00:15:04.117975 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.117939    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dbe86e4-c5f4-41c5-861a-fc624da2f122" path="/var/lib/kubelet/pods/3dbe86e4-c5f4-41c5-861a-fc624da2f122/volumes"
Jul 29 00:15:04.176935 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.176896    2564 generic.go:356] "Generic (PLEG): container finished" podID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0" containerID="b600d450ff4ea4c9b8bdf1f2aa1a514a587e8b4ce509925ad89e3dd9b049b2cb" exitCode=0
Jul 29 00:15:04.177331 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.176980    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210698256" event={"ID":"35e05bea-de04-43cd-bd5a-c1916e4d0fe0","Type":"ContainerDied","Data":"b600d450ff4ea4c9b8bdf1f2aa1a514a587e8b4ce509925ad89e3dd9b049b2cb"}
Jul 29 00:15:04.178378 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.178357    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"
Jul 29 00:15:04.178506 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.178371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj" event={"ID":"936068a2-27a7-49aa-8a22-1410d8d069b4","Type":"ContainerDied","Data":"1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35"}
Jul 29 00:15:04.178506 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.178426    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c75253df65b95e3b643ff0a5ef6d69f2a33506d8577467c56a1e7bd30093d35"
Jul 29 00:15:04.179790 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.179765    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b"
Jul 29 00:15:04.179893 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.179799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754735-n5k9b" event={"ID":"fac817f8-bcac-44c6-abe8-2ca130146abb","Type":"ContainerDied","Data":"6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2"}
Jul 29 00:15:04.179893 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.179846    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6aaaa3586a3393c027b8ec3098190af34533c45f7082207b850421395e2d80c2"
Jul 29 00:15:04.181675 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.181654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc" event={"ID":"36a27f93-4155-495a-911e-1467eacc303d","Type":"ContainerDied","Data":"ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f"}
Jul 29 00:15:04.181783 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.181694    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecde0d78a6d47659f4d6bc5509d44dc97f37f99488888cbbf4dd7b982599286f"
Jul 29 00:15:04.181783 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:04.181665    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"
Jul 29 00:15:05.218805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.218721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:05.331361 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.331315    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:05.351486 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.351453    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210698256_35e05bea-de04-43cd-bd5a-c1916e4d0fe0/test-curl-7401-140364210698256/0.log"
Jul 29 00:15:05.376699 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.376665    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:05.381006 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.380969    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210698256"]
Jul 29 00:15:05.438847 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.438758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwhjj\" (UniqueName: \"kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj\") pod \"35e05bea-de04-43cd-bd5a-c1916e4d0fe0\" (UID: \"35e05bea-de04-43cd-bd5a-c1916e4d0fe0\") "
Jul 29 00:15:05.440896 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.440870    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj" (OuterVolumeSpecName: "kube-api-access-kwhjj") pod "35e05bea-de04-43cd-bd5a-c1916e4d0fe0" (UID: "35e05bea-de04-43cd-bd5a-c1916e4d0fe0"). InnerVolumeSpecName "kube-api-access-kwhjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:05.499374 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499315    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:05.499887 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499868    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac817f8-bcac-44c6-abe8-2ca130146abb" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499889    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac817f8-bcac-44c6-abe8-2ca130146abb" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499911    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936068a2-27a7-49aa-8a22-1410d8d069b4" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499919    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="936068a2-27a7-49aa-8a22-1410d8d069b4" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499931    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a27f93-4155-495a-911e-1467eacc303d" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499940    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a27f93-4155-495a-911e-1467eacc303d" containerName="cleanup"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499952    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:05.499978 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.499962    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:05.500303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.500059    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0" containerName="test-curl-7401-140364210698256"
Jul 29 00:15:05.500303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.500075    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a27f93-4155-495a-911e-1467eacc303d" containerName="cleanup"
Jul 29 00:15:05.500303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.500086    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fac817f8-bcac-44c6-abe8-2ca130146abb" containerName="cleanup"
Jul 29 00:15:05.500303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.500100    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="936068a2-27a7-49aa-8a22-1410d8d069b4" containerName="cleanup"
Jul 29 00:15:05.503422 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.503402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:05.509217 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.509191    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:05.539924 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.539897    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwhjj\" (UniqueName: \"kubernetes.io/projected/35e05bea-de04-43cd-bd5a-c1916e4d0fe0-kube-api-access-kwhjj\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:15:05.641005 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.640963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dlfl\" (UniqueName: \"kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl\") pod \"test-curl-7401-140364210699264\" (UID: \"e348d584-5ef0-4848-9335-11eb722d9750\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:05.741949 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.741848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dlfl\" (UniqueName: \"kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl\") pod \"test-curl-7401-140364210699264\" (UID: \"e348d584-5ef0-4848-9335-11eb722d9750\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:05.751043 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.751010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dlfl\" (UniqueName: \"kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl\") pod \"test-curl-7401-140364210699264\" (UID: \"e348d584-5ef0-4848-9335-11eb722d9750\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:05.815513 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.815473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:05.961321 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:05.961290    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:05.962306 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:05.962281    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode348d584_5ef0_4848_9335_11eb722d9750.slice/crio-333fd86906cd951337b15fbb20790ec450300ad70eb4258522b6a380d402b123 WatchSource:0}: Error finding container 333fd86906cd951337b15fbb20790ec450300ad70eb4258522b6a380d402b123: Status 404 returned error can't find the container with id 333fd86906cd951337b15fbb20790ec450300ad70eb4258522b6a380d402b123
Jul 29 00:15:06.115884 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:06.115849    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e05bea-de04-43cd-bd5a-c1916e4d0fe0" path="/var/lib/kubelet/pods/35e05bea-de04-43cd-bd5a-c1916e4d0fe0/volumes"
Jul 29 00:15:06.195107 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:06.195070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210699264" event={"ID":"e348d584-5ef0-4848-9335-11eb722d9750","Type":"ContainerStarted","Data":"333fd86906cd951337b15fbb20790ec450300ad70eb4258522b6a380d402b123"}
Jul 29 00:15:06.196288 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:06.196272    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210698256"
Jul 29 00:15:06.196405 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:06.196317    2564 scope.go:117] "RemoveContainer" containerID="b600d450ff4ea4c9b8bdf1f2aa1a514a587e8b4ce509925ad89e3dd9b049b2cb"
Jul 29 00:15:07.204422 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:07.204381    2564 generic.go:356] "Generic (PLEG): container finished" podID="e348d584-5ef0-4848-9335-11eb722d9750" containerID="5d40dbbdacb92c26cefce77a35b71dc8a0d8a1ed738aa44f6c108c101132f92b" exitCode=0
Jul 29 00:15:07.204865 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:07.204446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210699264" event={"ID":"e348d584-5ef0-4848-9335-11eb722d9750","Type":"ContainerDied","Data":"5d40dbbdacb92c26cefce77a35b71dc8a0d8a1ed738aa44f6c108c101132f92b"}
Jul 29 00:15:08.353454 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.353430    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:08.372003 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.371976    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210699264_e348d584-5ef0-4848-9335-11eb722d9750/test-curl-7401-140364210699264/0.log"
Jul 29 00:15:08.395048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.395010    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:08.401268 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.401241    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:08.471660 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.471564    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dlfl\" (UniqueName: \"kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl\") pod \"e348d584-5ef0-4848-9335-11eb722d9750\" (UID: \"e348d584-5ef0-4848-9335-11eb722d9750\") "
Jul 29 00:15:08.473681 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.473648    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl" (OuterVolumeSpecName: "kube-api-access-8dlfl") pod "e348d584-5ef0-4848-9335-11eb722d9750" (UID: "e348d584-5ef0-4848-9335-11eb722d9750"). InnerVolumeSpecName "kube-api-access-8dlfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:08.536061 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.536028    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:08.536525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.536510    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e348d584-5ef0-4848-9335-11eb722d9750" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:08.536573 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.536526    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e348d584-5ef0-4848-9335-11eb722d9750" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:08.536607 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.536601    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e348d584-5ef0-4848-9335-11eb722d9750" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:08.539729 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.539713    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:08.546223 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.546190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:08.572448 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.572413    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dlfl\" (UniqueName: \"kubernetes.io/projected/e348d584-5ef0-4848-9335-11eb722d9750-kube-api-access-8dlfl\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:15:08.673255 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.673217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gklp9\" (UniqueName: \"kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9\") pod \"test-curl-7401-140364210699264\" (UID: \"68178305-3aea-45aa-aa66-0e1b56271980\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:08.774262 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.774166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gklp9\" (UniqueName: \"kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9\") pod \"test-curl-7401-140364210699264\" (UID: \"68178305-3aea-45aa-aa66-0e1b56271980\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:08.782735 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.782703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gklp9\" (UniqueName: \"kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9\") pod \"test-curl-7401-140364210699264\" (UID: \"68178305-3aea-45aa-aa66-0e1b56271980\") " pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:08.850956 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:08.850919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:09.196663 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:09.196631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:09.197418 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:09.197393    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68178305_3aea_45aa_aa66_0e1b56271980.slice/crio-452f13f38de83e06318f4b6da608fd0dba61e4cb39d776536b78e3af4e60906a WatchSource:0}: Error finding container 452f13f38de83e06318f4b6da608fd0dba61e4cb39d776536b78e3af4e60906a: Status 404 returned error can't find the container with id 452f13f38de83e06318f4b6da608fd0dba61e4cb39d776536b78e3af4e60906a
Jul 29 00:15:09.218530 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:09.218504    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="333fd86906cd951337b15fbb20790ec450300ad70eb4258522b6a380d402b123"
Jul 29 00:15:09.218680 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:09.218533    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:09.219922 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:09.219894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210699264" event={"ID":"68178305-3aea-45aa-aa66-0e1b56271980","Type":"ContainerStarted","Data":"452f13f38de83e06318f4b6da608fd0dba61e4cb39d776536b78e3af4e60906a"}
Jul 29 00:15:09.230525 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:09.230498    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7401-140364210699264" oldPodUID="e348d584-5ef0-4848-9335-11eb722d9750" podUID="68178305-3aea-45aa-aa66-0e1b56271980"
Jul 29 00:15:10.116727 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:10.116689    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e348d584-5ef0-4848-9335-11eb722d9750" path="/var/lib/kubelet/pods/e348d584-5ef0-4848-9335-11eb722d9750/volumes"
Jul 29 00:15:10.231312 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:10.231276    2564 generic.go:356] "Generic (PLEG): container finished" podID="68178305-3aea-45aa-aa66-0e1b56271980" containerID="af6010e505176958014fbf2cf71b64e68423693682740298c1f59a5f839f3f6e" exitCode=0
Jul 29 00:15:10.231504 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:10.231372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7401-140364210699264" event={"ID":"68178305-3aea-45aa-aa66-0e1b56271980","Type":"ContainerDied","Data":"af6010e505176958014fbf2cf71b64e68423693682740298c1f59a5f839f3f6e"}
Jul 29 00:15:11.379974 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.379947    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:11.405995 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.405953    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7401-140364210699264_68178305-3aea-45aa-aa66-0e1b56271980/test-curl-7401-140364210699264/0.log"
Jul 29 00:15:11.431317 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.431289    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:11.437126 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.437101    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7401-140364210699264"]
Jul 29 00:15:11.501577 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.501493    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gklp9\" (UniqueName: \"kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9\") pod \"68178305-3aea-45aa-aa66-0e1b56271980\" (UID: \"68178305-3aea-45aa-aa66-0e1b56271980\") "
Jul 29 00:15:11.503628 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.503604    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9" (OuterVolumeSpecName: "kube-api-access-gklp9") pod "68178305-3aea-45aa-aa66-0e1b56271980" (UID: "68178305-3aea-45aa-aa66-0e1b56271980"). InnerVolumeSpecName "kube-api-access-gklp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:11.603272 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.603232    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gklp9\" (UniqueName: \"kubernetes.io/projected/68178305-3aea-45aa-aa66-0e1b56271980-kube-api-access-gklp9\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:15:11.999505 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:11.999460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:12.115868 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:12.115835    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68178305-3aea-45aa-aa66-0e1b56271980" path="/var/lib/kubelet/pods/68178305-3aea-45aa-aa66-0e1b56271980/volumes"
Jul 29 00:15:12.244830 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:12.244802    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7401-140364210699264"
Jul 29 00:15:12.245012 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:12.244843    2564 scope.go:117] "RemoveContainer" containerID="af6010e505176958014fbf2cf71b64e68423693682740298c1f59a5f839f3f6e"
Jul 29 00:15:12.270230 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:12.270196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:12.448491 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:12.448458    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:15.506430 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.506393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:15.595802 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.595763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:15.602007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.601974    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:15:15.602550 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.602532    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68178305-3aea-45aa-aa66-0e1b56271980" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:15.602550 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.602552    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68178305-3aea-45aa-aa66-0e1b56271980" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:15.602690 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.602651    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68178305-3aea-45aa-aa66-0e1b56271980" containerName="test-curl-7401-140364210699264"
Jul 29 00:15:15.607270 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.607246    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.610412 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.610391    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 29 00:15:15.610541 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.610414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f318ab-serving-cert\""
Jul 29 00:15:15.616544 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.616513    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:15:15.698892 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.698857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:15.745195 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.745158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s9dp\" (UniqueName: \"kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.745428 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.745203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.745428 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.745237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.846915 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.846876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s9dp\" (UniqueName: \"kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.847071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.846930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.847071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.846980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.847165 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:15:15.847096    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-serving-cert: secret "maas-api-e2e-ipp-a-f318ab-serving-cert" not found
Jul 29 00:15:15.847165 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:15:15.847159    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls podName:aa3d0a8f-3691-45fb-964a-fc2d5702958f nodeName:}" failed. No retries permitted until 2026-07-29 00:15:16.347143177 +0000 UTC m=+3216.874192418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls") pod "maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" (UID: "aa3d0a8f-3691-45fb-964a-fc2d5702958f") : secret "maas-api-e2e-ipp-a-f318ab-serving-cert" not found
Jul 29 00:15:15.847707 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.847689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.856524 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.856504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s9dp\" (UniqueName: \"kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:15.936810 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:15.936773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:16.303679 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.303648    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:16.352225 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.352191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:16.354908 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.354879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:16.399864 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.399833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:16.520910 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.520821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:16.665856 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:16.665829    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:15:16.667748 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:16.667703    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3d0a8f_3691_45fb_964a_fc2d5702958f.slice/crio-6466eeaadd1907c4954da12729b4c1e187f2a77aa7debe425db924b8d54f640c WatchSource:0}: Error finding container 6466eeaadd1907c4954da12729b4c1e187f2a77aa7debe425db924b8d54f640c: Status 404 returned error can't find the container with id 6466eeaadd1907c4954da12729b4c1e187f2a77aa7debe425db924b8d54f640c
Jul 29 00:15:17.279368 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:17.279315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" event={"ID":"aa3d0a8f-3691-45fb-964a-fc2d5702958f","Type":"ContainerStarted","Data":"6466eeaadd1907c4954da12729b4c1e187f2a77aa7debe425db924b8d54f640c"}
Jul 29 00:15:18.287203 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:18.287163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" event={"ID":"aa3d0a8f-3691-45fb-964a-fc2d5702958f","Type":"ContainerStarted","Data":"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"}
Jul 29 00:15:18.287595 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:18.287296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:18.309680 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:18.309627    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" podStartSLOduration=2.47285019 podStartE2EDuration="3.30961062s" totalImagesPullingTime="836.76043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:18.306795844 +0000 UTC m=+3218.833845109" watchObservedRunningTime="2026-07-29 00:15:18.30961062 +0000 UTC m=+3218.836659882"
Jul 29 00:15:24.299317 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:24.299290    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:15:40.510367 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:40.510310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:41.209065 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.209032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:41.474935 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.474857    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:15:41.480258 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.480233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.482848 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.482828    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-eab07a-openshift-default-dockercfg-kmfzb\""
Jul 29 00:15:41.490815 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.490792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:15:41.512884 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.512853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:41.599593 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599654    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599929 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599929 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.599929 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.600038 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.599955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw926\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.700768 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.700731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.700936 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.700778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.700995 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.700975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701094 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701146 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701197 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw926\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701246 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701304 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701380 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701442 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701442 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701548 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701548 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.701654 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.701634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.703543 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.703521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.703750 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.703734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.715816 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.715789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.715945 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.715854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw926\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926\") pod \"e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") " pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.794304 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.794272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:41.812576 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.812548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:41.941241 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.941210    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:15:41.942502 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:41.942474    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658b83a7_3dc5_4877_b35c_75094d6623fa.slice/crio-67dfbd75f9975751b0162114e4a0809407ccf16c1b1933af50c392db784f120f WatchSource:0}: Error finding container 67dfbd75f9975751b0162114e4a0809407ccf16c1b1933af50c392db784f120f: Status 404 returned error can't find the container with id 67dfbd75f9975751b0162114e4a0809407ccf16c1b1933af50c392db784f120f
Jul 29 00:15:41.944855 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.944827    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:41.944936 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.944897    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:41.944936 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:41.944924    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:42.440402 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:42.440324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerStarted","Data":"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"}
Jul 29 00:15:42.440611 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:42.440426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerStarted","Data":"67dfbd75f9975751b0162114e4a0809407ccf16c1b1933af50c392db784f120f"}
Jul 29 00:15:42.463386 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:42.463323    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podStartSLOduration=1.463307726 podStartE2EDuration="1.463307726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:42.460526198 +0000 UTC m=+3242.987575461" watchObservedRunningTime="2026-07-29 00:15:42.463307726 +0000 UTC m=+3242.990356989"
Jul 29 00:15:42.794399 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:42.794336    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:43.801137 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:43.801097    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:44.454022 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.453979    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:44.455054 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.455032    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:44.455736 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.455701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:44.536647 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.536615    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:15:44.540610 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.540588    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.543319 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.543298    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-eab07a-serving-cert\""
Jul 29 00:15:44.551949 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.551923    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:15:44.565136 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.565099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:44.633582 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.633534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.633795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.633595    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpvbs\" (UniqueName: \"kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.633795 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.633630    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.662000 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.661960    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:44.735454 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.735307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.735454 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.735376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpvbs\" (UniqueName: \"kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.735454 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.735402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.735774 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:15:44.735557    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-serving-cert: secret "maas-api-e2e-ipp-b-eab07a-serving-cert" not found
Jul 29 00:15:44.735774 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:15:44.735616    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls podName:3baf63dc-dc15-489e-b2fe-f7316645f9c0 nodeName:}" failed. No retries permitted until 2026-07-29 00:15:45.235599901 +0000 UTC m=+3245.762649142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls") pod "maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" (UID: "3baf63dc-dc15-489e-b2fe-f7316645f9c0") : secret "maas-api-e2e-ipp-b-eab07a-serving-cert" not found
Jul 29 00:15:44.736181 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.736155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.746305 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.746271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpvbs\" (UniqueName: \"kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:44.874777 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:44.874743    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:45.105144 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.105113    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:45.216539 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.216465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:15:45.240883 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.240842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:45.243435 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.243401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:45.454253 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.454157    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:45.605284 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:45.605255    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:15:45.606627 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:15:45.606599    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3baf63dc_dc15_489e_b2fe_f7316645f9c0.slice/crio-8b95c8aad6822f5e4af1ae75a868ccbcbe83646d1265198dd96fc08164ac9c98 WatchSource:0}: Error finding container 8b95c8aad6822f5e4af1ae75a868ccbcbe83646d1265198dd96fc08164ac9c98: Status 404 returned error can't find the container with id 8b95c8aad6822f5e4af1ae75a868ccbcbe83646d1265198dd96fc08164ac9c98
Jul 29 00:15:46.468876 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:46.468840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" event={"ID":"3baf63dc-dc15-489e-b2fe-f7316645f9c0","Type":"ContainerStarted","Data":"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"}
Jul 29 00:15:46.469403 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:46.468900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" event={"ID":"3baf63dc-dc15-489e-b2fe-f7316645f9c0","Type":"ContainerStarted","Data":"8b95c8aad6822f5e4af1ae75a868ccbcbe83646d1265198dd96fc08164ac9c98"}
Jul 29 00:15:46.489206 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:46.489151    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" podStartSLOduration=2.059165093 podStartE2EDuration="2.489134869s" totalImagesPullingTime="429.969776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:46.486789828 +0000 UTC m=+3247.013839092" watchObservedRunningTime="2026-07-29 00:15:46.489134869 +0000 UTC m=+3247.016184132"
Jul 29 00:15:47.474629 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:47.474598    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:53.489369 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:53.489317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:15:57.540168 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.540134    2564 generic.go:356] "Generic (PLEG): container finished" podID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerID="9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba" exitCode=0
Jul 29 00:15:57.540612 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.540164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerDied","Data":"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"}
Jul 29 00:15:57.540675 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.540662    2564 scope.go:117] "RemoveContainer" containerID="9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"
Jul 29 00:15:57.541799 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.541768    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:57.541901 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.541836    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:57.541901 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:57.541867    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:58.547745 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:58.547713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerStarted","Data":"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"}
Jul 29 00:15:58.794734 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:58.794687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:58.796210 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:58.796181    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Jul 29 00:15:58.796368 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:58.796230    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Jul 29 00:15:59.454064 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:59.454024    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:15:59.794940 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:59.794909    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Jul 29 00:15:59.795383 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:15:59.794965    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Jul 29 00:16:00.795464 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:00.795429    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Jul 29 00:16:00.795888 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:00.795491    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Jul 29 00:16:01.794987 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:01.794946    2564 patch_prober.go:28] interesting pod/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Jul 29 00:16:01.795185 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:01.795029    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Jul 29 00:16:02.798913 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:02.798882    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:16:02.800020 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:02.799999    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:16:05.551565 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:05.551526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:16:09.336451 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:09.336412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:16:16.236816 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:16.236783    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:16:18.939720 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:18.939682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:16:40.399219 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:40.399189    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:16:40.407699 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:40.407668    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:16:40.408141 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:40.408118    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:16:40.416013 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:40.415986    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44zbm_083244cb-7b2a-4780-8636-faf88f1304da/ovn-acl-logging/0.log"
Jul 29 00:16:54.037807 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:16:54.037771    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:03.374503 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:03.374428    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:17:03.765378 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:03.765269    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:04.070461 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:04.070423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:04.315707 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:04.315678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.065580 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.065540    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.165135 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.165102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.340392 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.340281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.465634 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.465597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.856181 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.856146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:07.944863 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:07.944830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:19.424415 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:19.424373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:32.044314 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.044278    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:32.700985 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.700952    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"]
Jul 29 00:17:32.704844 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.704826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:32.707585 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.707561    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:17:32.710564 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.710536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"]
Jul 29 00:17:32.826391 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.826326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8\" (UID: \"d17e51d5-bf62-4898-a71e-dc9b9c124e94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:32.927706 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.927673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8\" (UID: \"d17e51d5-bf62-4898-a71e-dc9b9c124e94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:32.928296 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:32.928274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8\" (UID: \"d17e51d5-bf62-4898-a71e-dc9b9c124e94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:33.018909 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:33.018819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:33.161883 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:33.161854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"]
Jul 29 00:17:33.163130 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:17:33.163104    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17e51d5_bf62_4898_a71e_dc9b9c124e94.slice/crio-6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1 WatchSource:0}: Error finding container 6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1: Status 404 returned error can't find the container with id 6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1
Jul 29 00:17:33.217087 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:33.217056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8" event={"ID":"d17e51d5-bf62-4898-a71e-dc9b9c124e94","Type":"ContainerStarted","Data":"6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1"}
Jul 29 00:17:34.224677 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:34.224590    2564 generic.go:356] "Generic (PLEG): container finished" podID="d17e51d5-bf62-4898-a71e-dc9b9c124e94" containerID="016e40325c2074e22d1a7741debbf3797dab936145de4bb8cf1520a85904807d" exitCode=0
Jul 29 00:17:34.225044 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:34.224668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8" event={"ID":"d17e51d5-bf62-4898-a71e-dc9b9c124e94","Type":"ContainerDied","Data":"016e40325c2074e22d1a7741debbf3797dab936145de4bb8cf1520a85904807d"}
Jul 29 00:17:35.367779 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:35.367749    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:35.452629 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:35.452598    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca\") pod \"d17e51d5-bf62-4898-a71e-dc9b9c124e94\" (UID: \"d17e51d5-bf62-4898-a71e-dc9b9c124e94\") "
Jul 29 00:17:35.452963 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:35.452937    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d17e51d5-bf62-4898-a71e-dc9b9c124e94" (UID: "d17e51d5-bf62-4898-a71e-dc9b9c124e94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:17:35.553363 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:35.553310    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17e51d5-bf62-4898-a71e-dc9b9c124e94-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:17:36.240656 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:36.240625    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"
Jul 29 00:17:36.240656 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:36.240648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8" event={"ID":"d17e51d5-bf62-4898-a71e-dc9b9c124e94","Type":"ContainerDied","Data":"6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1"}
Jul 29 00:17:36.240858 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:36.240701    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6954dda544fec0b1ddd506e049232c5fa9fb52c85b0e6d21cc94a20c38aaa2b1"
Jul 29 00:17:37.797397 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:37.797338    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:37.863010 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:37.862974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:37.926663 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:37.926632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:37.964444 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:37.964386    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"]
Jul 29 00:17:37.967234 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:37.967204    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e26439-2hfb8"]
Jul 29 00:17:38.095313 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:38.095211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:38.116219 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:38.116187    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17e51d5-bf62-4898-a71e-dc9b9c124e94" path="/var/lib/kubelet/pods/d17e51d5-bf62-4898-a71e-dc9b9c124e94/volumes"
Jul 29 00:17:38.397520 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:38.397430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:41.447317 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:41.447256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:41.548168 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:41.548132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:44.097317 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:44.097280    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:44.161441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:44.161406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:44.343740 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:44.343710    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:44.569034 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:44.568981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:17:47.942335 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.942292    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"]
Jul 29 00:17:47.942791 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.942767    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17e51d5-bf62-4898-a71e-dc9b9c124e94" containerName="revoke-keys"
Jul 29 00:17:47.942791 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.942779    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17e51d5-bf62-4898-a71e-dc9b9c124e94" containerName="revoke-keys"
Jul 29 00:17:47.942880 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.942869    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17e51d5-bf62-4898-a71e-dc9b9c124e94" containerName="revoke-keys"
Jul 29 00:17:47.947221 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.947204    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:47.949918 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.949893    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:17:47.952700 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:47.952673    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"]
Jul 29 00:17:48.088289 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:48.088247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq\" (UID: \"036e6d7f-000a-4e1b-b154-74bd325d6f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:48.189192 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:48.189155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq\" (UID: \"036e6d7f-000a-4e1b-b154-74bd325d6f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:48.189820 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:48.189799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq\" (UID: \"036e6d7f-000a-4e1b-b154-74bd325d6f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:48.261003 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:48.260914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:48.404433 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:48.404403    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"]
Jul 29 00:17:48.405569 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:17:48.405538    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036e6d7f_000a_4e1b_b154_74bd325d6f62.slice/crio-eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9 WatchSource:0}: Error finding container eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9: Status 404 returned error can't find the container with id eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9
Jul 29 00:17:49.327413 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:49.327310    2564 generic.go:356] "Generic (PLEG): container finished" podID="036e6d7f-000a-4e1b-b154-74bd325d6f62" containerID="2fbdbc0a7c3154de9a320b1f5fce4e112482a4a6cfb470cf678bfd4d077286b6" exitCode=0
Jul 29 00:17:49.327413 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:49.327381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq" event={"ID":"036e6d7f-000a-4e1b-b154-74bd325d6f62","Type":"ContainerDied","Data":"2fbdbc0a7c3154de9a320b1f5fce4e112482a4a6cfb470cf678bfd4d077286b6"}
Jul 29 00:17:49.327822 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:49.327437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq" event={"ID":"036e6d7f-000a-4e1b-b154-74bd325d6f62","Type":"ContainerStarted","Data":"eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9"}
Jul 29 00:17:50.468891 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:50.468857    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:50.508372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:50.508321    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca\") pod \"036e6d7f-000a-4e1b-b154-74bd325d6f62\" (UID: \"036e6d7f-000a-4e1b-b154-74bd325d6f62\") "
Jul 29 00:17:50.508773 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:50.508735    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "036e6d7f-000a-4e1b-b154-74bd325d6f62" (UID: "036e6d7f-000a-4e1b-b154-74bd325d6f62"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:17:50.609555 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:50.609473    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e6d7f-000a-4e1b-b154-74bd325d6f62-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:17:51.341144 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:51.341110    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"
Jul 29 00:17:51.341318 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:51.341140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq" event={"ID":"036e6d7f-000a-4e1b-b154-74bd325d6f62","Type":"ContainerDied","Data":"eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9"}
Jul 29 00:17:51.341318 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:17:51.341191    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eab7613e4a06b60a2e60517ffc69d22bdd44647517fb3b9949251f794e48a7b9"
Jul 29 00:18:03.567099 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.567059    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:18:03.567662 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.567317    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" podUID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" containerName="maas-api" containerID="cri-o://72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc" gracePeriod=30
Jul 29 00:18:03.579016 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.578988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:03.705687 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.705632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:03.752217 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.752183    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"]
Jul 29 00:18:03.754940 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.754910    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f318ab-9dmsq"]
Jul 29 00:18:03.834545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.834518    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:18:03.920305 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.920272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:03.941433 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.941407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca\") pod \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") "
Jul 29 00:18:03.941609 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.941487    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") pod \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") "
Jul 29 00:18:03.941609 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.941580    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s9dp\" (UniqueName: \"kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp\") pod \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\" (UID: \"aa3d0a8f-3691-45fb-964a-fc2d5702958f\") "
Jul 29 00:18:03.941826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.941804    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa3d0a8f-3691-45fb-964a-fc2d5702958f" (UID: "aa3d0a8f-3691-45fb-964a-fc2d5702958f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:18:03.943692 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.943662    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp" (OuterVolumeSpecName: "kube-api-access-2s9dp") pod "aa3d0a8f-3691-45fb-964a-fc2d5702958f" (UID: "aa3d0a8f-3691-45fb-964a-fc2d5702958f"). InnerVolumeSpecName "kube-api-access-2s9dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:18:03.943813 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:03.943786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa3d0a8f-3691-45fb-964a-fc2d5702958f" (UID: "aa3d0a8f-3691-45fb-964a-fc2d5702958f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:18:04.043223 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.043184    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa3d0a8f-3691-45fb-964a-fc2d5702958f-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:18:04.043223 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.043222    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s9dp\" (UniqueName: \"kubernetes.io/projected/aa3d0a8f-3691-45fb-964a-fc2d5702958f-kube-api-access-2s9dp\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:18:04.043223 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.043232    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa3d0a8f-3691-45fb-964a-fc2d5702958f-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:18:04.117562 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.117481    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="036e6d7f-000a-4e1b-b154-74bd325d6f62" path="/var/lib/kubelet/pods/036e6d7f-000a-4e1b-b154-74bd325d6f62/volumes"
Jul 29 00:18:04.136886 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.136828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:04.430854 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.430775    2564 generic.go:356] "Generic (PLEG): container finished" podID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" containerID="72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc" exitCode=0
Jul 29 00:18:04.430854 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.430824    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"
Jul 29 00:18:04.431048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.430862    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" event={"ID":"aa3d0a8f-3691-45fb-964a-fc2d5702958f","Type":"ContainerDied","Data":"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"}
Jul 29 00:18:04.431048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.430915    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf" event={"ID":"aa3d0a8f-3691-45fb-964a-fc2d5702958f","Type":"ContainerDied","Data":"6466eeaadd1907c4954da12729b4c1e187f2a77aa7debe425db924b8d54f640c"}
Jul 29 00:18:04.431048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.430936    2564 scope.go:117] "RemoveContainer" containerID="72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"
Jul 29 00:18:04.440680 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.440658    2564 scope.go:117] "RemoveContainer" containerID="72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"
Jul 29 00:18:04.440987 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:18:04.440967    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc\": container with ID starting with 72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc not found: ID does not exist" containerID="72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"
Jul 29 00:18:04.441026 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.441000    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc"} err="failed to get container status \"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc\": rpc error: code = NotFound desc = could not find container \"72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc\": container with ID starting with 72f3cd59e6d615ceb6aee8c35da5f5b44cd56106e3d28a08b1c2260bd6b6c1cc not found: ID does not exist"
Jul 29 00:18:04.456655 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.456624    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:18:04.461854 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:04.461825    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f318ab-58748945d7-5bgpf"]
Jul 29 00:18:06.116334 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:06.116297    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" path="/var/lib/kubelet/pods/aa3d0a8f-3691-45fb-964a-fc2d5702958f/volumes"
Jul 29 00:18:09.581289 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:09.581254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:46.254129 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:46.254087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:46.423294 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:46.423260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:52.134515 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.134475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:52.202649 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.202605    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:52.433697 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.433559    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:52.673378 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.673334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:18:52.797372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797327    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"]
Jul 29 00:18:52.797815 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797802    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="036e6d7f-000a-4e1b-b154-74bd325d6f62" containerName="revoke-keys"
Jul 29 00:18:52.797861 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797817    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="036e6d7f-000a-4e1b-b154-74bd325d6f62" containerName="revoke-keys"
Jul 29 00:18:52.797861 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797832    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" containerName="maas-api"
Jul 29 00:18:52.797861 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797838    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" containerName="maas-api"
Jul 29 00:18:52.797959 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797920    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa3d0a8f-3691-45fb-964a-fc2d5702958f" containerName="maas-api"
Jul 29 00:18:52.797959 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.797933    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="036e6d7f-000a-4e1b-b154-74bd325d6f62" containerName="revoke-keys"
Jul 29 00:18:52.802453 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.802435    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:52.805453 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.805400    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:18:52.809418 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.809383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"]
Jul 29 00:18:52.937954 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:52.937915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx\" (UID: \"80ac4df4-0f38-40cb-a524-df8ce7de3ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:53.039485 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:53.039448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx\" (UID: \"80ac4df4-0f38-40cb-a524-df8ce7de3ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:53.040064 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:53.040043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx\" (UID: \"80ac4df4-0f38-40cb-a524-df8ce7de3ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:53.114785 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:53.114700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:53.474493 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:53.474454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"]
Jul 29 00:18:53.744428 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:53.744310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx" event={"ID":"80ac4df4-0f38-40cb-a524-df8ce7de3ab8","Type":"ContainerStarted","Data":"bcdfcef8a39ffc5c39081370984969d10dbe213bb0a6343b31bc2c7164fd390e"}
Jul 29 00:18:54.753490 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:54.753453    2564 generic.go:356] "Generic (PLEG): container finished" podID="80ac4df4-0f38-40cb-a524-df8ce7de3ab8" containerID="522da7e74c82bdb4c8dd140e51113827bc5f8d6893f5a3304da6612cb0c8d422" exitCode=0
Jul 29 00:18:54.753906 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:54.753546    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx" event={"ID":"80ac4df4-0f38-40cb-a524-df8ce7de3ab8","Type":"ContainerDied","Data":"522da7e74c82bdb4c8dd140e51113827bc5f8d6893f5a3304da6612cb0c8d422"}
Jul 29 00:18:55.911780 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:55.911751    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:56.071154 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.071115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca\") pod \"80ac4df4-0f38-40cb-a524-df8ce7de3ab8\" (UID: \"80ac4df4-0f38-40cb-a524-df8ce7de3ab8\") "
Jul 29 00:18:56.071526 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.071496    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80ac4df4-0f38-40cb-a524-df8ce7de3ab8" (UID: "80ac4df4-0f38-40cb-a524-df8ce7de3ab8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:18:56.172659 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.172616    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac4df4-0f38-40cb-a524-df8ce7de3ab8-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:18:56.768437 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.768404    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"
Jul 29 00:18:56.768618 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.768407    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx" event={"ID":"80ac4df4-0f38-40cb-a524-df8ce7de3ab8","Type":"ContainerDied","Data":"bcdfcef8a39ffc5c39081370984969d10dbe213bb0a6343b31bc2c7164fd390e"}
Jul 29 00:18:56.768618 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:18:56.768542    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcdfcef8a39ffc5c39081370984969d10dbe213bb0a6343b31bc2c7164fd390e"
Jul 29 00:19:03.243457 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.243417    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:19:03.243948 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.243628    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerName="maas-api" containerID="cri-o://97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7" gracePeriod=30
Jul 29 00:19:03.269603 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.269571    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:03.369863 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.369814    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:03.445234 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.445171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:03.445449 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.445339    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"]
Jul 29 00:19:03.450113 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.450082    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-eab07a-jlpjx"]
Jul 29 00:19:03.512287 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.512260    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:19:03.518953 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.518924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:03.647066 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.647030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") pod \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") "
Jul 29 00:19:03.647066 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.647074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca\") pod \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") "
Jul 29 00:19:03.647322 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.647118    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpvbs\" (UniqueName: \"kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs\") pod \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\" (UID: \"3baf63dc-dc15-489e-b2fe-f7316645f9c0\") "
Jul 29 00:19:03.647559 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.647528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3baf63dc-dc15-489e-b2fe-f7316645f9c0" (UID: "3baf63dc-dc15-489e-b2fe-f7316645f9c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:03.649406 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.649378    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3baf63dc-dc15-489e-b2fe-f7316645f9c0" (UID: "3baf63dc-dc15-489e-b2fe-f7316645f9c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:19:03.649506 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.649406    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs" (OuterVolumeSpecName: "kube-api-access-gpvbs") pod "3baf63dc-dc15-489e-b2fe-f7316645f9c0" (UID: "3baf63dc-dc15-489e-b2fe-f7316645f9c0"). InnerVolumeSpecName "kube-api-access-gpvbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:03.748542 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.748444    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpvbs\" (UniqueName: \"kubernetes.io/projected/3baf63dc-dc15-489e-b2fe-f7316645f9c0-kube-api-access-gpvbs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:03.748542 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.748477    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3baf63dc-dc15-489e-b2fe-f7316645f9c0-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:03.748542 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.748489    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3baf63dc-dc15-489e-b2fe-f7316645f9c0-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:03.816254 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.816219    2564 generic.go:356] "Generic (PLEG): container finished" podID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerID="97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7" exitCode=0
Jul 29 00:19:03.816441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.816271    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"
Jul 29 00:19:03.816441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.816305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" event={"ID":"3baf63dc-dc15-489e-b2fe-f7316645f9c0","Type":"ContainerDied","Data":"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"}
Jul 29 00:19:03.816441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.816377    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" event={"ID":"3baf63dc-dc15-489e-b2fe-f7316645f9c0","Type":"ContainerDied","Data":"8b95c8aad6822f5e4af1ae75a868ccbcbe83646d1265198dd96fc08164ac9c98"}
Jul 29 00:19:03.816441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.816407    2564 scope.go:117] "RemoveContainer" containerID="97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"
Jul 29 00:19:03.826967 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.826948    2564 scope.go:117] "RemoveContainer" containerID="97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"
Jul 29 00:19:03.827217 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:19:03.827198    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7\": container with ID starting with 97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7 not found: ID does not exist" containerID="97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"
Jul 29 00:19:03.827264 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.827227    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7"} err="failed to get container status \"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7\": rpc error: code = NotFound desc = could not find container \"97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7\": container with ID starting with 97ed979bbcc1622014b5874540e3718be821619ac2ae3bb63f32da3d86e358e7 not found: ID does not exist"
Jul 29 00:19:03.837105 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.837079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:03.838818 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.838798    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:19:03.841540 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:03.841521    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4"]
Jul 29 00:19:04.116650 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:04.116613    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" path="/var/lib/kubelet/pods/3baf63dc-dc15-489e-b2fe-f7316645f9c0/volumes"
Jul 29 00:19:04.116979 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:04.116966    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ac4df4-0f38-40cb-a524-df8ce7de3ab8" path="/var/lib/kubelet/pods/80ac4df4-0f38-40cb-a524-df8ce7de3ab8/volumes"
Jul 29 00:19:06.481888 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:06.481828    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eab07a-584f8dc7fc-llnr4" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.95:8443/health\": context deadline exceeded"
Jul 29 00:19:09.685614 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:09.685582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:09.787862 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:09.787823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:09.935516 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:09.935473    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:19:09.935777 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:09.935715    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy" containerID="cri-o://0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9" gracePeriod=30
Jul 29 00:19:09.987849 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:09.987812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:10.174962 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:10.174925    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:15.203312 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.203251    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:19:15.247733 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247704    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.247898 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247741    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.247898 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw926\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.247898 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247824    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.247898 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.248106 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247907    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.248106 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.247977    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.248106 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:15.248292 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248265    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.248428 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket\") pod \"658b83a7-3dc5-4877-b35c-75094d6623fa\" (UID: \"658b83a7-3dc5-4877-b35c-75094d6623fa\") "
Jul 29 00:19:15.248805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:15.248805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248764    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:15.248805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248796    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.249008 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248815    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.249008 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.248969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:15.249291 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.249260    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:15.250166 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.250145    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:19:15.250283 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.250267    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:15.250532 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.250511    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926" (OuterVolumeSpecName: "kube-api-access-jw926") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "kube-api-access-jw926". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:15.250718 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.250701    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "658b83a7-3dc5-4877-b35c-75094d6623fa" (UID: "658b83a7-3dc5-4877-b35c-75094d6623fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:15.349551 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349511    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349551 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349544    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349551 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349559    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw926\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-kube-api-access-jw926\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349572    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349584    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/658b83a7-3dc5-4877-b35c-75094d6623fa-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349595    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.349805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.349605    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/658b83a7-3dc5-4877-b35c-75094d6623fa-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:15.898603 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.898564    2564 generic.go:356] "Generic (PLEG): container finished" podID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerID="0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9" exitCode=0
Jul 29 00:19:15.898796 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.898616    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"
Jul 29 00:19:15.898796 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.898652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerDied","Data":"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"}
Jul 29 00:19:15.898796 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.898715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck" event={"ID":"658b83a7-3dc5-4877-b35c-75094d6623fa","Type":"ContainerDied","Data":"67dfbd75f9975751b0162114e4a0809407ccf16c1b1933af50c392db784f120f"}
Jul 29 00:19:15.898796 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.898748    2564 scope.go:117] "RemoveContainer" containerID="0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"
Jul 29 00:19:15.909806 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.909779    2564 scope.go:117] "RemoveContainer" containerID="9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"
Jul 29 00:19:15.918744 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.918721    2564 scope.go:117] "RemoveContainer" containerID="0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"
Jul 29 00:19:15.919018 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:19:15.918999    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9\": container with ID starting with 0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9 not found: ID does not exist" containerID="0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"
Jul 29 00:19:15.919057 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.919028    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9"} err="failed to get container status \"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9\": rpc error: code = NotFound desc = could not find container \"0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9\": container with ID starting with 0e0c7d15fa1ba6b32b7ebe5dfc7a19264619e635fecf56588478a6db8a69ffc9 not found: ID does not exist"
Jul 29 00:19:15.919057 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.919049    2564 scope.go:117] "RemoveContainer" containerID="9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"
Jul 29 00:19:15.919321 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:19:15.919303    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba\": container with ID starting with 9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba not found: ID does not exist" containerID="9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"
Jul 29 00:19:15.920078 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.919328    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba"} err="failed to get container status \"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba\": rpc error: code = NotFound desc = could not find container \"9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba\": container with ID starting with 9f96ea4737d6003d1c7834916aa2b7b804c2165abc732047b0205eeddcab1eba not found: ID does not exist"
Jul 29 00:19:15.923671 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.923649    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:19:15.930562 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:15.930540    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-eab07a-openshift-default-dbb4fcf48-bvvck"]
Jul 29 00:19:16.117766 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:16.117731    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" path="/var/lib/kubelet/pods/658b83a7-3dc5-4877-b35c-75094d6623fa/volumes"
Jul 29 00:19:20.849967 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:20.849924    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"]
Jul 29 00:19:20.852463 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:20.852440    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-daedc5-29754735-dwxfc"]
Jul 29 00:19:20.886930 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:20.886889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:20.957914 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:20.957886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:21.052635 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:21.052600    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:21.411218 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:21.411180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:22.116654 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:22.116617    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a27f93-4155-495a-911e-1467eacc303d" path="/var/lib/kubelet/pods/36a27f93-4155-495a-911e-1467eacc303d/volumes"
Jul 29 00:19:27.333441 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.333400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:27.446955 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.446919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:27.573157 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.573121    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:19:27.573455 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.573423    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy" containerID="cri-o://5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380" gracePeriod=30
Jul 29 00:19:27.600564 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.600487    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:27.821893 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.821860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:27.935037 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.934963    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"]
Jul 29 00:19:27.935456 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935441    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935460    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935476    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935481    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935491    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ac4df4-0f38-40cb-a524-df8ce7de3ab8" containerName="revoke-keys"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935503    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ac4df4-0f38-40cb-a524-df8ce7de3ab8" containerName="revoke-keys"
Jul 29 00:19:27.935517 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935513    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerName="maas-api"
Jul 29 00:19:27.935705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935520    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerName="maas-api"
Jul 29 00:19:27.935705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935612    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ac4df4-0f38-40cb-a524-df8ce7de3ab8" containerName="revoke-keys"
Jul 29 00:19:27.935705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935626    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935633    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="658b83a7-3dc5-4877-b35c-75094d6623fa" containerName="istio-proxy"
Jul 29 00:19:27.935705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.935639    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3baf63dc-dc15-489e-b2fe-f7316645f9c0" containerName="maas-api"
Jul 29 00:19:27.938897 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.938881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:27.941499 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.941472    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:19:27.945312 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:27.945284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"]
Jul 29 00:19:28.080843 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.080801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt\" (UID: \"03b27297-1c4b-43da-8bca-78125769658c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:28.181393 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.181327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt\" (UID: \"03b27297-1c4b-43da-8bca-78125769658c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:28.182003 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.181979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt\" (UID: \"03b27297-1c4b-43da-8bca-78125769658c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:28.251772 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.251684    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:28.405007 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.404976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"]
Jul 29 00:19:28.406187 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:19:28.406164    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b27297_1c4b_43da_8bca_78125769658c.slice/crio-7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5 WatchSource:0}: Error finding container 7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5: Status 404 returned error can't find the container with id 7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5
Jul 29 00:19:28.983620 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.983446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt" event={"ID":"03b27297-1c4b-43da-8bca-78125769658c","Type":"ContainerStarted","Data":"2c221b1547c20651a308c994b44de216449e00d0f7e1d075156d741354e227bb"}
Jul 29 00:19:28.983620 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.983519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt" event={"ID":"03b27297-1c4b-43da-8bca-78125769658c","Type":"ContainerStarted","Data":"7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5"}
Jul 29 00:19:28.999897 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:28.999845    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt" podStartSLOduration=1.999828909 podStartE2EDuration="1.999828909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:19:28.997731946 +0000 UTC m=+3469.524781208" watchObservedRunningTime="2026-07-29 00:19:28.999828909 +0000 UTC m=+3469.526878172"
Jul 29 00:19:29.992900 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:29.992865    2564 generic.go:356] "Generic (PLEG): container finished" podID="03b27297-1c4b-43da-8bca-78125769658c" containerID="2c221b1547c20651a308c994b44de216449e00d0f7e1d075156d741354e227bb" exitCode=0
Jul 29 00:19:29.993308 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:29.992952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt" event={"ID":"03b27297-1c4b-43da-8bca-78125769658c","Type":"ContainerDied","Data":"2c221b1547c20651a308c994b44de216449e00d0f7e1d075156d741354e227bb"}
Jul 29 00:19:31.146616 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:31.146584    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:31.210397 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:31.210336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca\") pod \"03b27297-1c4b-43da-8bca-78125769658c\" (UID: \"03b27297-1c4b-43da-8bca-78125769658c\") "
Jul 29 00:19:31.210692 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:31.210671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03b27297-1c4b-43da-8bca-78125769658c" (UID: "03b27297-1c4b-43da-8bca-78125769658c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:31.311433 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:31.311399    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b27297-1c4b-43da-8bca-78125769658c-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:32.007486 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.007452    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"
Jul 29 00:19:32.007745 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.007486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt" event={"ID":"03b27297-1c4b-43da-8bca-78125769658c","Type":"ContainerDied","Data":"7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5"}
Jul 29 00:19:32.007745 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.007571    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7991c77ab077f61b7a5ef27565a6a0e5ee324070457238866299600852b3fed5"
Jul 29 00:19:32.841313 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.841245    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:19:32.927636 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927593    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927679    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj4qj\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927738    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927767    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927803    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.927852 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.928170 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.928170 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927967    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket\") pod \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\" (UID: \"b251d90e-3ac2-4e8e-9524-1998bfb255c7\") "
Jul 29 00:19:32.928170 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.927988    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:32.928330 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928168    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:32.928330 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:32.928330 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:32.928538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928468    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:32.928538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928488    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:32.928538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928501    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:32.928538 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928514    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:32.928755 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.928703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:32.930034 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.929997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:19:32.930169 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.930117    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:32.930310 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.930293    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj" (OuterVolumeSpecName: "kube-api-access-nj4qj") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "kube-api-access-nj4qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:32.930382 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:32.930327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b251d90e-3ac2-4e8e-9524-1998bfb255c7" (UID: "b251d90e-3ac2-4e8e-9524-1998bfb255c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:33.016074 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.016036    2564 generic.go:356] "Generic (PLEG): container finished" podID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerID="5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380" exitCode=0
Jul 29 00:19:33.016244 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.016090    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"
Jul 29 00:19:33.016244 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.016124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerDied","Data":"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"}
Jul 29 00:19:33.016244 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.016184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt" event={"ID":"b251d90e-3ac2-4e8e-9524-1998bfb255c7","Type":"ContainerDied","Data":"303524c22174e8f5c8099b7a8c378b477cd444be2a3ad84d9394cadb4e678839"}
Jul 29 00:19:33.016244 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.016208    2564 scope.go:117] "RemoveContainer" containerID="5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"
Jul 29 00:19:33.027006 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.026986    2564 scope.go:117] "RemoveContainer" containerID="102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"
Jul 29 00:19:33.028941 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.028923    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:33.028941 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.028943    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:33.029071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.028951    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj4qj\" (UniqueName: \"kubernetes.io/projected/b251d90e-3ac2-4e8e-9524-1998bfb255c7-kube-api-access-nj4qj\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:33.029071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.028960    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:33.029071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.028968    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b251d90e-3ac2-4e8e-9524-1998bfb255c7-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\""
Jul 29 00:19:33.037452 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.037431    2564 scope.go:117] "RemoveContainer" containerID="5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"
Jul 29 00:19:33.037750 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:19:33.037729    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380\": container with ID starting with 5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380 not found: ID does not exist" containerID="5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"
Jul 29 00:19:33.037809 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.037760    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380"} err="failed to get container status \"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380\": rpc error: code = NotFound desc = could not find container \"5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380\": container with ID starting with 5293008f1c7db87179b2c36c22fdb8ffa830b2f187fdf0a36a14bb42feb38380 not found: ID does not exist"
Jul 29 00:19:33.037809 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.037778    2564 scope.go:117] "RemoveContainer" containerID="102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"
Jul 29 00:19:33.038004 ip-10-0-135-12 kubenswrapper[2564]: E0729 00:19:33.037987    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83\": container with ID starting with 102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83 not found: ID does not exist" containerID="102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"
Jul 29 00:19:33.038051 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.038010    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83"} err="failed to get container status \"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83\": rpc error: code = NotFound desc = could not find container \"102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83\": container with ID starting with 102b32af1ded4300405cd1d5ad15b474e72979800d7b896a55e9f8d759938f83 not found: ID does not exist"
Jul 29 00:19:33.039531 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.039511    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:19:33.044566 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:33.044546    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-daedc5-openshift-default-6d6b4d87dd-7zbvt"]
Jul 29 00:19:34.116364 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:34.116321    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" path="/var/lib/kubelet/pods/b251d90e-3ac2-4e8e-9524-1998bfb255c7/volumes"
Jul 29 00:19:38.570071 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.570021    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"]
Jul 29 00:19:38.578882 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.578840    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fa6130-29754735-vshfj"]
Jul 29 00:19:38.584758 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.584724    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:38.657676 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.657644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:38.749743 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.749707    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"]
Jul 29 00:19:38.751113 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.751061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:38.752298 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.752275    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-fa6130-zz6dt"]
Jul 29 00:19:38.807336 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:38.807293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:39.129432 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:39.129384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:40.115669 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:40.115627    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03b27297-1c4b-43da-8bca-78125769658c" path="/var/lib/kubelet/pods/03b27297-1c4b-43da-8bca-78125769658c/volumes"
Jul 29 00:19:40.116137 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:40.116043    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="936068a2-27a7-49aa-8a22-1410d8d069b4" path="/var/lib/kubelet/pods/936068a2-27a7-49aa-8a22-1410d8d069b4/volumes"
Jul 29 00:19:45.039574 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:45.039533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:45.139995 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:45.139963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:45.296316 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:45.296231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:45.523683 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:45.523648    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:19:45.945822 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:45.945774    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bd7dd74bc-lj5j5_f910a034-218b-48eb-97da-5308277a0ec3/authorino/0.log"
Jul 29 00:19:51.286566 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:51.286530    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zsgtr_e62753f4-f908-4cc4-a6f1-293529dd0665/manager/0.log"
Jul 29 00:19:51.401587 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:51.401553    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bprkh_9774aa96-fd1c-4820-9018-271302c43913/manager/0.log"
Jul 29 00:19:51.642470 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:51.642439    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lnv9v_5be4c05b-ebc6-46b6-8945-1fa180e0a124/server/0.log"
Jul 29 00:19:51.754623 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:51.754588    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xrpcd_bf59ff25-6226-45cf-8c79-9002e905ff9c/manager/0.log"
Jul 29 00:19:51.994837 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:51.994761    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-pjvw2_a76b2b36-e116-4399-bbb3-0d21e7971da4/copy-manifests/0.log"
Jul 29 00:19:52.001564 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:52.001533    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-pjvw2_a76b2b36-e116-4399-bbb3-0d21e7971da4/manager/0.log"
Jul 29 00:19:52.577033 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:52.576996    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754705-52lc9_19a13a59-c6f1-4d66-bd21-8dbf9d95033e/cleanup/0.log"
Jul 29 00:19:52.688242 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:52.688208    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754720-mkr6w_b11608e0-22d6-4e63-85a3-03e989de51b6/cleanup/0.log"
Jul 29 00:19:52.797907 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:52.797877    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754735-n5k9b_fac817f8-bcac-44c6-abe8-2ca130146abb/cleanup/0.log"
Jul 29 00:19:52.912378 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:52.912272    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-d4fl9_faee699a-6256-41e8-8c52-84964b027f3f/postgres/0.log"
Jul 29 00:19:53.678372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.678317    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/extract/0.log"
Jul 29 00:19:53.683551 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.683528    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/util/0.log"
Jul 29 00:19:53.688931 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.688914    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/pull/0.log"
Jul 29 00:19:53.804029 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.803997    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/util/0.log"
Jul 29 00:19:53.809680 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.809655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/pull/0.log"
Jul 29 00:19:53.814962 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.814944    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/extract/0.log"
Jul 29 00:19:53.924865 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.924837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/util/0.log"
Jul 29 00:19:53.929819 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.929752    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/pull/0.log"
Jul 29 00:19:53.935805 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:53.935783    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/extract/0.log"
Jul 29 00:19:54.043808 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.043778    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/extract/0.log"
Jul 29 00:19:54.049057 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.049038    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/util/0.log"
Jul 29 00:19:54.054330 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.054310    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/pull/0.log"
Jul 29 00:19:54.166204 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.166174    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bd7dd74bc-lj5j5_f910a034-218b-48eb-97da-5308277a0ec3/authorino/0.log"
Jul 29 00:19:54.279769 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.279694    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-49px7_f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8/manager/0.log"
Jul 29 00:19:54.611282 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.611254    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n2zv7_10e14de4-2c8e-4165-ac71-f608c25cdcfb/registry-server/0.log"
Jul 29 00:19:54.729728 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.729694    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q4dg5_37f2bd5c-f48b-45c4-a8c2-eb8d15f923db/manager/0.log"
Jul 29 00:19:54.958809 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:54.958727    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v7469_dc572114-b48d-403e-af1d-5f6e42dfdefc/manager/0.log"
Jul 29 00:19:55.304237 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:55.304206    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm5xph_3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e/istio-proxy/0.log"
Jul 29 00:19:55.628913 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:55.628837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f8c77c69-9lspv_82d87be9-0d29-45cf-bb90-95ef38a03dde/kube-auth-proxy/0.log"
Jul 29 00:19:55.861378 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:55.861336    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xmsls_11154eb5-663f-4585-ab9c-723b1db23531/payload-pre-processing/0.log"
Jul 29 00:19:55.994132 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:55.994027    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:19:56.110301 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:56.110270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5f6bbfdc-9q6tr_f78be253-2630-4747-9957-79ad1db992af/router/0.log"
Jul 29 00:19:56.564183 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:19:56.564152    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8d7mp_0367daa4-4783-4206-97e0-c4c01be7c89b/main/0.log"
Jul 29 00:20:02.559869 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.559638    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bwncz/must-gather-dw4ht"]
Jul 29 00:20:02.560540 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560516    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.560540 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560542    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.560710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560554    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03b27297-1c4b-43da-8bca-78125769658c" containerName="revoke-keys"
Jul 29 00:20:02.560710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560562    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="03b27297-1c4b-43da-8bca-78125769658c" containerName="revoke-keys"
Jul 29 00:20:02.560710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560593    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.560710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560602    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.560916 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560722    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="03b27297-1c4b-43da-8bca-78125769658c" containerName="revoke-keys"
Jul 29 00:20:02.560916 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560736    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.560916 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.560748    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b251d90e-3ac2-4e8e-9524-1998bfb255c7" containerName="istio-proxy"
Jul 29 00:20:02.564262 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.564238    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.566874 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.566845    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwncz\"/\"openshift-service-ca.crt\""
Jul 29 00:20:02.566874 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.566865    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwncz\"/\"kube-root-ca.crt\""
Jul 29 00:20:02.568217 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.568194    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bwncz\"/\"default-dockercfg-42lcq\""
Jul 29 00:20:02.571989 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.571784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/must-gather-dw4ht"]
Jul 29 00:20:02.588421 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.588377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:20:02.615031 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.614989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhf2g\" (UniqueName: \"kubernetes.io/projected/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-kube-api-access-xhf2g\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.615206 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.615117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-must-gather-output\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.716073 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.716034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-must-gather-output\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.716326 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.716092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhf2g\" (UniqueName: \"kubernetes.io/projected/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-kube-api-access-xhf2g\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.716423 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.716402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-must-gather-output\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.724421 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.724382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhf2g\" (UniqueName: \"kubernetes.io/projected/6256b2fd-61b1-43e2-b5fa-4b7f824370b9-kube-api-access-xhf2g\") pod \"must-gather-dw4ht\" (UID: \"6256b2fd-61b1-43e2-b5fa-4b7f824370b9\") " pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:02.875015 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:02.874919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/must-gather-dw4ht"
Jul 29 00:20:03.030949 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:03.030916    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/must-gather-dw4ht"]
Jul 29 00:20:03.032281 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:20:03.032258    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6256b2fd_61b1_43e2_b5fa_4b7f824370b9.slice/crio-5fce8adb08dce9ff2719b275fdc16fc554e6f7ef1a2ba5a70d73d2275ab5ce09 WatchSource:0}: Error finding container 5fce8adb08dce9ff2719b275fdc16fc554e6f7ef1a2ba5a70d73d2275ab5ce09: Status 404 returned error can't find the container with id 5fce8adb08dce9ff2719b275fdc16fc554e6f7ef1a2ba5a70d73d2275ab5ce09
Jul 29 00:20:03.034058 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:03.034040    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:20:03.206257 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:03.206167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/must-gather-dw4ht" event={"ID":"6256b2fd-61b1-43e2-b5fa-4b7f824370b9","Type":"ContainerStarted","Data":"5fce8adb08dce9ff2719b275fdc16fc554e6f7ef1a2ba5a70d73d2275ab5ce09"}
Jul 29 00:20:04.085172 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:04.085089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log" limit=1024
Jul 29 00:20:04.216835 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:04.216801    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/must-gather-dw4ht" event={"ID":"6256b2fd-61b1-43e2-b5fa-4b7f824370b9","Type":"ContainerStarted","Data":"6387a326e7b41b88f9f73b8e66c94c8de2b5c318ba73d77226b9de2497636372"}
Jul 29 00:20:04.216835 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:04.216842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/must-gather-dw4ht" event={"ID":"6256b2fd-61b1-43e2-b5fa-4b7f824370b9","Type":"ContainerStarted","Data":"991d8a02275d660c76d08eef66671c6a6b85467e5c1ec687c1c735bbaefac3be"}
Jul 29 00:20:04.233996 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:04.233936    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bwncz/must-gather-dw4ht" podStartSLOduration=1.5116732640000001 podStartE2EDuration="2.233915618s" totalImagesPullingTime="722.242354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:20:04.231655388 +0000 UTC m=+3504.758704655" watchObservedRunningTime="2026-07-29 00:20:04.233915618 +0000 UTC m=+3504.760964885"
Jul 29 00:20:05.316757 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:05.316723    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f4k6f_afaeb2da-9369-4aea-b14f-baf82476d5d0/global-pull-secret-syncer/0.log"
Jul 29 00:20:05.405833 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:05.405781    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4bvx6_c96c4a66-6319-41f3-b325-25bbdbff0d52/konnectivity-agent/0.log"
Jul 29 00:20:05.513460 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:05.513425    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-12.ec2.internal_39abfa21be29f5a114b5174162890ec9/haproxy/0.log"
Jul 29 00:20:09.931545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:09.931511    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/extract/0.log"
Jul 29 00:20:09.956059 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:09.956020    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/util/0.log"
Jul 29 00:20:09.978473 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:09.978443    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t85pr_e0e60e5d-3582-477a-b064-6787cffa20f1/pull/0.log"
Jul 29 00:20:10.009111 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.008932    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/extract/0.log"
Jul 29 00:20:10.031159 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.031131    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/util/0.log"
Jul 29 00:20:10.054074 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.053997    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqj87_d6340f7e-7567-4c9f-8e30-677866e5ab3c/pull/0.log"
Jul 29 00:20:10.107759 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.107722    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/extract/0.log"
Jul 29 00:20:10.149970 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.149944    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/util/0.log"
Jul 29 00:20:10.173548 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.173511    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ftgxr_34f9c0cc-dca2-4907-9ba6-49b9ca8b3448/pull/0.log"
Jul 29 00:20:10.200048 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.199933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/extract/0.log"
Jul 29 00:20:10.218775 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.218746    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/util/0.log"
Jul 29 00:20:10.239203 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.239170    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17bmbf_a4b67969-083c-4302-bba9-04e1a0530f23/pull/0.log"
Jul 29 00:20:10.795725 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.795438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bd7dd74bc-lj5j5_f910a034-218b-48eb-97da-5308277a0ec3/authorino/0.log"
Jul 29 00:20:10.824385 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.824305    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-49px7_f2bd4c9b-9c95-43b5-9209-6da15f4eeeb8/manager/0.log"
Jul 29 00:20:10.928046 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:10.928010    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n2zv7_10e14de4-2c8e-4165-ac71-f608c25cdcfb/registry-server/0.log"
Jul 29 00:20:11.159338 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:11.158889    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q4dg5_37f2bd5c-f48b-45c4-a8c2-eb8d15f923db/manager/0.log"
Jul 29 00:20:11.283145 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:11.283111    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v7469_dc572114-b48d-403e-af1d-5f6e42dfdefc/manager/0.log"
Jul 29 00:20:13.015888 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.015854    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/alertmanager/0.log"
Jul 29 00:20:13.038208 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.038178    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/config-reloader/0.log"
Jul 29 00:20:13.063196 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.063167    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/kube-rbac-proxy-web/0.log"
Jul 29 00:20:13.085651 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.085587    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/kube-rbac-proxy/0.log"
Jul 29 00:20:13.109259 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.109204    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/kube-rbac-proxy-metric/0.log"
Jul 29 00:20:13.136528 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.136495    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/prom-label-proxy/0.log"
Jul 29 00:20:13.160695 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.160655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_59286e52-1968-4b6d-a03b-1185417b4adc/init-config-reloader/0.log"
Jul 29 00:20:13.237017 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.236982    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ghshp_4c8d1e37-645f-43c8-a575-edc9bb4fa514/kube-state-metrics/0.log"
Jul 29 00:20:13.252705 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.252673    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ghshp_4c8d1e37-645f-43c8-a575-edc9bb4fa514/kube-rbac-proxy-main/0.log"
Jul 29 00:20:13.276738 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.276633    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ghshp_4c8d1e37-645f-43c8-a575-edc9bb4fa514/kube-rbac-proxy-self/0.log"
Jul 29 00:20:13.305587 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.305556    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86598459fb-rfn8p_dad8d7b8-3ea0-4a40-8022-4adbacaa7acc/metrics-server/0.log"
Jul 29 00:20:13.436369 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.436327    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w84mq_3502deed-e2b6-4b35-a431-b774b5c5e9ae/node-exporter/0.log"
Jul 29 00:20:13.456672 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.456642    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w84mq_3502deed-e2b6-4b35-a431-b774b5c5e9ae/kube-rbac-proxy/0.log"
Jul 29 00:20:13.482206 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.482179    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w84mq_3502deed-e2b6-4b35-a431-b774b5c5e9ae/init-textfile/0.log"
Jul 29 00:20:13.611114 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.611086    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mkjdj_7e62e3d3-a42f-486e-bb31-6ecb6132e700/kube-rbac-proxy-main/0.log"
Jul 29 00:20:13.632575 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.632546    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mkjdj_7e62e3d3-a42f-486e-bb31-6ecb6132e700/kube-rbac-proxy-self/0.log"
Jul 29 00:20:13.655961 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.655933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-mkjdj_7e62e3d3-a42f-486e-bb31-6ecb6132e700/openshift-state-metrics/0.log"
Jul 29 00:20:13.846314 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.846267    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-sjw6s_aef6b6fa-d0ed-4dff-869d-9b8dec920a2a/prometheus-operator/0.log"
Jul 29 00:20:13.875098 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.875066    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-sjw6s_aef6b6fa-d0ed-4dff-869d-9b8dec920a2a/kube-rbac-proxy/0.log"
Jul 29 00:20:13.906110 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:13.906083    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-xs845_ec64a69f-f83c-4799-9452-6afc6af4e254/prometheus-operator-admission-webhook/0.log"
Jul 29 00:20:15.722702 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.722644    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"]
Jul 29 00:20:15.731301 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.731268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.734214 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.734182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"]
Jul 29 00:20:15.864404 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.864332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xbb\" (UniqueName: \"kubernetes.io/projected/9cbb4808-03dd-4429-b692-6cf7598698ba-kube-api-access-r5xbb\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.864709 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.864691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-sys\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.864834 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.864819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-lib-modules\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.864972 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.864959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-proc\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.865101 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.865089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-podres\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966167 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-lib-modules\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-proc\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-podres\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966372 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-lib-modules\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-proc\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-podres\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xbb\" (UniqueName: \"kubernetes.io/projected/9cbb4808-03dd-4429-b692-6cf7598698ba-kube-api-access-r5xbb\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966545 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-sys\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.966777 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.966590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cbb4808-03dd-4429-b692-6cf7598698ba-sys\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:15.974339 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.974245    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/2.log"
Jul 29 00:20:15.981366 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.981321    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l26sg_fef84116-f628-4535-a269-2e5bebfe6200/console-operator/3.log"
Jul 29 00:20:15.984882 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:15.984851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xbb\" (UniqueName: \"kubernetes.io/projected/9cbb4808-03dd-4429-b692-6cf7598698ba-kube-api-access-r5xbb\") pod \"perf-node-gather-daemonset-7d92b\" (UID: \"9cbb4808-03dd-4429-b692-6cf7598698ba\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:16.050045 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:16.050002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:16.256568 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:16.256477    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"]
Jul 29 00:20:16.260113 ip-10-0-135-12 kubenswrapper[2564]: W0729 00:20:16.260076    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9cbb4808_03dd_4429_b692_6cf7598698ba.slice/crio-e7fbe2bf58277e0540ed0eb7482736e190967bff4daa9fde9ffc4e34dcfb413c WatchSource:0}: Error finding container e7fbe2bf58277e0540ed0eb7482736e190967bff4daa9fde9ffc4e34dcfb413c: Status 404 returned error can't find the container with id e7fbe2bf58277e0540ed0eb7482736e190967bff4daa9fde9ffc4e34dcfb413c
Jul 29 00:20:16.339105 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:16.337826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b" event={"ID":"9cbb4808-03dd-4429-b692-6cf7598698ba","Type":"ContainerStarted","Data":"e7fbe2bf58277e0540ed0eb7482736e190967bff4daa9fde9ffc4e34dcfb413c"}
Jul 29 00:20:16.540823 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:16.540796    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58997c4d99-88rlb_62c178a4-085d-4fcb-80d1-cfadfd6f2faf/console/0.log"
Jul 29 00:20:16.581056 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:16.581024    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-jhdmx_12a4e6aa-0230-4af6-9b71-b2a34df9d6a7/download-server/0.log"
Jul 29 00:20:17.346010 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:17.345977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b" event={"ID":"9cbb4808-03dd-4429-b692-6cf7598698ba","Type":"ContainerStarted","Data":"8a81281361f63ee653c2ae0d2b4ca47993a64242333899f00c4b0f2ec5101536"}
Jul 29 00:20:17.346486 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:17.346122    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:17.364687 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:17.364630    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b" podStartSLOduration=2.364612522 podStartE2EDuration="2.364612522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:20:17.360556816 +0000 UTC m=+3517.887606076" watchObservedRunningTime="2026-07-29 00:20:17.364612522 +0000 UTC m=+3517.891661785"
Jul 29 00:20:18.229815 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:18.229787    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6rt8g_d4cd0759-0e50-4f91-82b7-2d12fc26c4c9/dns/0.log"
Jul 29 00:20:18.248307 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:18.248272    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6rt8g_d4cd0759-0e50-4f91-82b7-2d12fc26c4c9/kube-rbac-proxy/0.log"
Jul 29 00:20:18.366717 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:18.366689    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zwzzw_78cf8882-071f-4e89-9728-13511310389b/dns-node-resolver/0.log"
Jul 29 00:20:18.861248 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:18.861208    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29754720-xswqb_6fdcae63-d249-4050-b85f-7d69f565f896/image-pruner/0.log"
Jul 29 00:20:18.948083 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:18.948012    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h2ts6_c7bac126-cd63-4eb0-9e70-77bd8be227f8/node-ca/0.log"
Jul 29 00:20:19.940252 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:19.940220    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm5xph_3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e/istio-proxy/0.log"
Jul 29 00:20:20.132110 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:20.132066    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f8c77c69-9lspv_82d87be9-0d29-45cf-bb90-95ef38a03dde/kube-auth-proxy/0.log"
Jul 29 00:20:20.219235 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:20.219147    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xmsls_11154eb5-663f-4585-ab9c-723b1db23531/payload-pre-processing/0.log"
Jul 29 00:20:20.250231 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:20.250202    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:20:20.270920 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:20.270887    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5f6bbfdc-9q6tr_f78be253-2630-4747-9957-79ad1db992af/router/0.log"
Jul 29 00:20:21.133738 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.133696    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zdkqg_2405c937-b80c-4c4c-ab91-bd9ee88da811/serve-healthcheck-canary/0.log"
Jul 29 00:20:21.605404 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.605373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-8gkcs_8f91cf43-36ec-47cb-870c-57d39b4b4de2/insights-operator/0.log"
Jul 29 00:20:21.606579 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.606482    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-8gkcs_8f91cf43-36ec-47cb-870c-57d39b4b4de2/insights-operator/1.log"
Jul 29 00:20:21.759034 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.759003    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmcnl_d9a684e9-2655-488b-bfc4-f8a7d89359f6/kube-rbac-proxy/0.log"
Jul 29 00:20:21.778784 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.778747    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmcnl_d9a684e9-2655-488b-bfc4-f8a7d89359f6/exporter/0.log"
Jul 29 00:20:21.798574 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:21.798534    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmcnl_d9a684e9-2655-488b-bfc4-f8a7d89359f6/extractor/0.log"
Jul 29 00:20:23.366784 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:23.366757    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-7d92b"
Jul 29 00:20:23.856724 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:23.856689    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zsgtr_e62753f4-f908-4cc4-a6f1-293529dd0665/manager/0.log"
Jul 29 00:20:24.194255 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:24.194154    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bprkh_9774aa96-fd1c-4820-9018-271302c43913/manager/0.log"
Jul 29 00:20:24.361892 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:24.361858    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lnv9v_5be4c05b-ebc6-46b6-8945-1fa180e0a124/server/0.log"
Jul 29 00:20:24.455913 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:24.455828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xrpcd_bf59ff25-6226-45cf-8c79-9002e905ff9c/manager/0.log"
Jul 29 00:20:24.522502 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:24.522462    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-pjvw2_a76b2b36-e116-4399-bbb3-0d21e7971da4/manager/0.log"
Jul 29 00:20:24.542303 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:24.542268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-pjvw2_a76b2b36-e116-4399-bbb3-0d21e7971da4/copy-manifests/0.log"
Jul 29 00:20:26.108486 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.108398    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bp2l2_5b37b1c8-7952-41f6-96f9-35b425a95622/openshift-lws-operator/0.log"
Jul 29 00:20:26.380540 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.380462    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm5xph_3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e/istio-proxy/0.log"
Jul 29 00:20:26.546690 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.546662    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f8c77c69-9lspv_82d87be9-0d29-45cf-bb90-95ef38a03dde/kube-auth-proxy/0.log"
Jul 29 00:20:26.615710 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.615659    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xmsls_11154eb5-663f-4585-ab9c-723b1db23531/payload-pre-processing/0.log"
Jul 29 00:20:26.642998 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.642911    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:20:26.661221 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:26.661175    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5f6bbfdc-9q6tr_f78be253-2630-4747-9957-79ad1db992af/router/0.log"
Jul 29 00:20:28.057742 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:28.057705    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm5xph_3b721ebe-c34e-4205-ac9a-d6aeb5e0a11e/istio-proxy/0.log"
Jul 29 00:20:28.230115 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:28.230085    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f8c77c69-9lspv_82d87be9-0d29-45cf-bb90-95ef38a03dde/kube-auth-proxy/0.log"
Jul 29 00:20:28.306392 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:28.306360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xmsls_11154eb5-663f-4585-ab9c-723b1db23531/payload-pre-processing/0.log"
Jul 29 00:20:28.334087 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:28.333969    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wrxds_3aa31e84-a9ac-4b9e-b2f4-2f6c3ce9539e/payload-processing/0.log"
Jul 29 00:20:28.357835 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:28.357804    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5f6bbfdc-9q6tr_f78be253-2630-4747-9957-79ad1db992af/router/0.log"
Jul 29 00:20:32.524618 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.524589    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/kube-multus-additional-cni-plugins/0.log"
Jul 29 00:20:32.544567 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.544545    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/egress-router-binary-copy/0.log"
Jul 29 00:20:32.565233 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.565190    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/cni-plugins/0.log"
Jul 29 00:20:32.584467 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.584440    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/bond-cni-plugin/0.log"
Jul 29 00:20:32.602826 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.602801    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/routeoverride-cni/0.log"
Jul 29 00:20:32.621455 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.621431    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/whereabouts-cni-bincopy/0.log"
Jul 29 00:20:32.639748 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:32.639729    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8cfrg_5f155a88-e612-4fbd-a32e-906d0e74ce6c/whereabouts-cni/0.log"
Jul 29 00:20:33.013642 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:33.013608    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gwzhj_ddee4c9a-6fe1-4dca-a5fa-f5607136a2c3/kube-multus/0.log"
Jul 29 00:20:33.103470 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:33.103359    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bw89j_43cc8c79-993f-463f-9b5e-8aaaa4a9d068/network-metrics-daemon/0.log"
Jul 29 00:20:33.133558 ip-10-0-135-12 kubenswrapper[2564]: I0729 00:20:33.133525    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bw89j_43cc8c79-993f-463f-9b5e-8aaaa4a9d068/kube-rbac-proxy/0.log"